Databricks Architect
Unison Group · Melbourne
Unison Group is looking for a Databricks Architect to lead the design and implementation of scalable, high-performance data platforms leveraging Databricks and Apache Spark. The successful candidate will collaborate with data engineers, scientists, and business teams to build data lakes, lakehouse architectures, and analytics solutions that drive business insights.
- Architect and implement Databricks-based data platforms and pipelines.
- Design end-to-end solutions for data ingestion, processing, and analytics using Databricks and Spark.
- Develop best practices for data engineering, governance, and security within the Databricks environment.
- Optimize compute and storage resources to balance cost and performance.
- Collaborate closely with cross-functional teams to understand requirements and translate them into scalable solutions.
Requirements
- Strong hands-on experience with Databricks, Apache Spark, and Delta Lake technologies.
- Proficiency in big data programming languages such as Python, Scala, and SQL.
- Experience designing and deploying data lakes, lakehouse architectures, and ETL/ELT pipelines on cloud platforms (AWS, Azure, or GCP).
- Deep understanding of data architecture, data modeling, and data governance.
- Excellent analytical, communication, and solution design skills.
- Prior experience working in agile, collaborative environments.