Intermediate Data Engineer

SUMMARY:

POSITION INFO:
Key Responsibilities Data Engineering & Platform Development Design, build, and maintain scalable data pipelines and workflows. Develop and optimise data processing solutions using Databricks and Apache Spark . Build and support end-to-end data integration and transformation processes. Ensure data solutions are reliable, efficient, and aligned with best-practice engineering standards. Cloud & Big Data Solutions Develop solutions within Azure cloud environments. Support the implementation and optimisation of modern data architectures. Work with large-scale datasets and distributed computing frameworks. Contribute to the ongoing evolution of the organisation's data platform capabilities. Data Pipeline Automation & CI\/CD Develop and maintain CI\/CD pipelines to support efficient deployment and release processes. Implement automation strategies that improve data delivery, quality, and operational efficiency. Support monitoring, testing, and optimisation of data engineering workflows. Collaboration & Solution Delivery Work closely with Data Engineers, Developers, Analysts, and Business Stakeholders to deliver high-quality data solutions. Participate in solution design discussions and technical planning. Contribute to the continuous improvement of data engineering practices and standards. Innovation & Continuous Learning Stay current with emerging technologies and trends within data engineering and cloud platforms. Identify opportunities to improve performance, scalability, and maintainability across data solutions. Contribute to knowledge sharing and technical excellence within the team. Requirements Essential Skills & Experience 3-5 years' experience in a Data Engineering, Backend Development, or similar technical role. Previous experience working as a Data Engineer . Strong experience with Databricks . Proven expertise in Apache Spark and cluster computing environments. Advanced Python skills, including mandatory PySpark experience. Strong SQL development skills. Experience developing and maintaining CI\/CD pipelines . Experience building scalable and reusable data pipelines. Solid software engineering and backend development foundations. Exposure to Azure cloud services and infrastructure . Advantageous Experience Microsoft Fabric experience. Medallion Architecture experience. Experience working with modern data warehousing and analytics environments. Exposure to cloud-native data engineering practices. What Success Looks Like High-performing and scalable data pipelines are delivered and maintained. Data workflows are optimised for reliability, performance, and cost efficiency. CI\/CD processes support efficient and consistent deployments. Strong collaboration with technical and business teams results in high-quality data solutions. Best practices for data engineering, cloud development, and automation are continuously adopted and improved. Apply now! For more IT jobs, please visit I also specialise in recruiting in the following: Software Development Data Engineer Data Analyst Infrastructure Architecture ... and more! If you have not had any response in two weeks, please consider the vacancy application unsuccessful. Your profile will be kept on our database for any other suitable roles \/ positions. For more information contact: Sfundo Shoba