Senior Data Engineer - Databricks

Perficient · Colombia

- Design, build, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of large-scale data.
- Develop and optimize data processing logic using PySpark on Databricks (Apache Spark).
- Implement ETL/ELT pipelines integrating data from multiple structured and semi-structured sources.
- Contribute to the design and implementation of lakehouse architectures (Delta Lake, Medallion architecture).
- Ensure data quality, reliability, performance, and observability across pipelines.
- Optimize Spark jobs through partitioning, caching, and performance tuning techniques.
- Collaborate with data architects, analysts, and business stakeholders to translate requirements into scalable data solutions.
- Implement best practices in CI/CD, version control, and pipeline automation.
- Support the evolution of modern data platforms and analytics capabilities.
- Work with orchestration tools (ADF, Airflow, or similar) to manage workflows.
- Communicate technical solutions and trade-offs effectively to both technical and non-technical stakeholders.

Perficient is the global AI and technology consulting firm disrupting the traditional consulting model. Powered by our 7,000+ advisors, engineers, and designers, Perficient implements AI-first solutions that break conventions and deliver outcomes that matter. Proudly serving clients that represent the world’s most innovative brands, and in collaboration with our powerful technology partner ecosystem, we bring deep industry expertise and data-driven design to redefine how businesses run and succeed. Perficient is different. For real. Learn more at perficient.com. We currently have a career opportunity for a Senior Data Engineer to join our team located in LatAm. We are looking for a highly skilled professional with strong expertise in modern data engineering and distributed processing, who will design and build scalable data solutions using Databricks, PySpark, and cloud-native technologies. You will work on data-driven platforms, enabling ingestion, transformation, and advanced analytics at scale while following best practices in data architecture, performance optimization, and data reliability.

More jobs at Perficient · Data & Analytics jobs in Colombia, Colombia