Senior Data Science Specialist
HCLTech · Sweden
Job Summary
We are looking for a Databricks expert with strong experience in platform setup, architecture, and AI/ML capabilities across Azure and/or GCP. The ideal candidate should bring hands-on expertise in building Databricks-based AI solutions, not just traditional data pipelines.
Key Responsibilities
- Design and implement Databricks architecture on Azure and/or GCP (workspace, clusters, networking, security).
- Lead platform setup, governance, and optimization (Unity Catalog, cost control, access management).
- Build and deploy AI/ML solutions using Databricks (MLflow, Feature Store, Model Serving).
- Develop GenAI / LLM solutions (RAG pipelines, embeddings, vector search).
- Integrate LLMs (Azure OpenAI / Vertex AI / other APIs) within Databricks.
- Deliver scalable Lakehouse architecture solutions.
- Collaborate with engineering and business teams for end-to-end AI use cases.
- Ensure performance tuning, reliability, and security of workloads
Skill Requirements
- Design and implement Databricks architecture on Azure and/or GCP (workspace, clusters, networking, security).
- Lead platform setup, governance, and optimization (Unity Catalog, cost control, access management).
- Build and deploy AI/ML solutions using Databricks (MLflow, Feature Store, Model Serving).
- Develop GenAI / LLM solutions (RAG pipelines, embeddings, vector search).
- Integrate LLMs (Azure OpenAI / Vertex AI / other APIs) within Databricks.
- Deliver scalable Lakehouse architecture solutions.
- Collaborate with engineering and business teams for end-to-end AI use cases.
- Ensure performance tuning, reliability, and security of workloads