Staff Data Engineer - Data COE
Pearson · Bengaluru
Job Title: Staff Data Engineer
About the Role
We are seeking an Analytics Engineer to design, build, and operate our analytics and automations as well as build of AI-powered automations and copilots using governed enterprise data. This role is responsible for delivering high-quality Power BI reporting, establishing and maintaining Microsoft Fabric and/or GCP BigQuery, and building business automations and applications using Python, Power Automate and Power Apps.
You will be part of the Cloud & Service Management organization helping to evolve our self-service analytics, scalable data architecture, and automations—while ensuring security, performance, and governance across the platform.
This is a hands-on role with ownership of both solution delivery and platform best practices.
Shift & Work Location: The candidate will work US shift timings and remotely for the initial 6 months. From month 7 onwards, the role transitions to UK/US business hours with a hybrid working model from the office.
Key Responsibilities
Analytics & Reporting
- Design, develop, and maintain reports and dashboards
- Build and optimize semantic models using strong dimensional modeling (star schema)
- Write and tune DAX measures with a focus on performance and usability
- Implement Power BI and Looker deployment pipelines and promote content across environments
Microsoft Fabric Platform
- Establish and maintain Microsoft Fabric architecture, including:
- Lakehouse and/or Warehouse
- Dataflows Gen2
- OneLake data organization
- Manage Fabric capacities, workspaces, and permissions
- Monitor performance, cost, and reliability of Fabric workloads
- Develop and maintain Python-based data transformations and notebooks within Fabric
- Use Python for data preparation, enrichment, validation, and advanced analytics
- Define and enforce data modeling and medallion architecture standards
Automation & Applications
- Build and maintain automation flows for business processes, approvals, and integrations
- Work with Dataverse, connectors, and security roles
- Implement error handling, logging, and operational support patterns
Platform Governance & Operations
- Define Dev/Test/Prod environment strategy for reporting and automation platform
- Implement Application Life best practices (solutions, pipelines, source control where applicable)
- Establish governance standards to prevent platform sprawl
- Partner with security and IT teams on access control and compliance
- Provide guidance and enablement to analysts and citizen developers
Collaboration & Leadership
- Translate business requirements into scalable technical solutions
- Contribute to platform roadmap and continuous improvement efforts
Agentic AI & ML Enablement
- Design and deliver agentic AI solutions that automate multi-step business workflows (tool use, planning, and human-in-the-loop approvals) using enterprise data and governed actions.
- Build RAG (retrieval-augmented generation) patterns over Fabric/OneLake (document ingestion, chunking, embeddings, retrieval evaluation) to power analytics copilots and self-service Q&A.
- Develop and operate ML pipelines (feature engineering, training, evaluation, batch/real-time inference) using Python and approved ML frameworks.
- Establish LLMOps/ModelOps practices: prompt/version control, offline evaluation, regression testing, monitoring (quality, drift, cost, latency), and safe rollback.
- Implement AI security and governance: data access controls, prompt/data leakage prevention, PII handling, model risk reviews, and audit logging for agent actions.
- Partner with stakeholders to identify high-value use cases and deliver measurable outcomes (time saved, defect reduction, SLA improvements).
Required Qualifications
- 5+ years of experience in analytics, BI, or data engineering roles
- 3+ years of hands-on Power BI development experience
- Strong experience with Microsoft Fabric (Lakehouse, Warehouse, Dataflows)
- Proficient in DAX, SQL, and data modeling
- Hands-on experience with:
- Power Automate (cloud flows, approvals, integrations)
- Power Apps (Canvas apps)
- Dataverse
- Hands-on Python experience delivering ML or GenAI solutions in production (notebooks-to-service, APIs, scheduled jobs, or integrated automations).
- Working knowledge of RAG concepts (embeddings, vector search, retrieval, grounding, evaluation).
- Experience implementing monitoring and testing for data/ML/GenAI systems (data quality checks, model/prompt evaluation, logging/telemetry).
- Experience managing environments, security, and deployments
- Strong understanding of data governance and analytics best practices
Preferred Qualifications
- Experience designing enterprise-scale analytics platforms
- Familiarity with Azure services (Azure SQL, Data Factory, Synapse)
- Familiarity with GCP BigQuery and Looker
- Experience with CI/CD concepts for Power BI and Looker
- Power Platform or Microsoft analytics certifications
- Experience working in a Center of Excellence (CoE) model
- Experience with Azure OpenAI / Azure AI Foundry (or equivalent) and enterprise deployment patterns.
- Experience with orchestration frameworks (e.g., Semantic Kernel, LangChain, Autogen) and tool/function calling.