SUMMARY:
-
POSITION INFO:
We are looking for a Databricks Certified Data Engineer to join our Cape Town-based client on a resource augmentation contract. The ideal candidate will have strong experience with the Azure Data ecosystem and advanced proficiency in Databricks and PySpark, with proven ability to build scalable, high-performance data pipelines and analytics solutions. Certification in Databricks is mandatory.
Key Responsibilities:
- Design, develop and maintain robust ETL/ELT pipelines using Azure Data Factory (ADF) and/or Fivetran
- Implement data processing workflows using Databricks, PySpark, and Delta Live Tables
- Build and optimise data transformation logic using Python and/or SQL
- Collaborate using Databricks notebooks and manage code versioning via Git
- Administer and monitor Azure-based environments, preferably with Terraform for infrastructure-as-code
- Apply data governance and lineage practices using Unity Catalog
- Work with structured and semi-structured data sourced from SAP systems
- Participate in data modelling, pipeline optimisation, and performance tuning
- Ensure high standards of data quality, reliability, and integrity
Minimum Requirements:
- Databricks Data Engineer Associate or Professional Certification (must be current/valid)
- 3+ years of experience in data engineering or related roles
- Strong working knowledge of Databricks, PySpark, and Apache Spark
- Proficiency in SQL and Python for data transformation and analysis
- Experience with Azure Data Factory and/or Fivetran
- Comfortable using Databricks Notebooks and Git for collaboration
- Experience with Terraform and Azure infrastructure administration
- Familiarity with Delta Lake, Delta Live Tables, and Unity Catalog
- Exposure to data ingestion or integration from SAP systems is advantageous
Soft Skills & Attributes:
- Strong problem-solving and analytical skills
- Ability to work independently and in cross-functional teams
- Excellent communication and documentation skills
- Agile mindset and adaptability to changing priorities
Interested candidates must be Databricks certified and available for contract engagement.
Please submit your CV along with proof of certification.
* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent
* If you have not received any feedback after 2 weeks, please consider you application as unsuccessful.
