Required Skills & Experience
- 3+ years of experience in Data Engineering or Data Integration.
- Hands-on experience with Databricks.
- Strong working experience with Apache Spark and PySpark.
- Good programming skills in Python.
- Strong SQL skills, including complex queries, joins, aggregations and transformations.
- Experience developing ETL/ELT pipelines.
- Experience integrating data using REST APIs and other API-based mechanisms.
- Working knowledge of Databricks Unity Catalog.
- Understanding of data governance, data quality and data management practices.
- Working knowledge of CI/CD pipelines and Git-based development.
- Good troubleshooting and problem-solving skills.
Strongly Preferred Skills
- Experience with Delta Lake and Lakehouse architecture.
- Exposure to Databricks Workflows and job scheduling.
- Experience with data validation and automated quality checks.
- Understanding of data lineage, metadata management and access controls.
- Exposure to cloud platforms such as AWS or Azure.
- Experience working in Agile/Scrum delivery environments.