Data Engineer

SUMMARY:

Our client is looking for an experienced Data Engineer to join a high-performing team focused on building scalable, cloud-native data solutions that enable enterprise-wide analytics and AI/ML capabilities. This role will be instrumental in designing, developing, and optimising robust data pipelines within an AWS ecosystem.

POSITION INFO:

Our client is looking for an experienced Data Engineer to join a high-performing team focused on building scalable, cloud-native data solutions that enable enterprise-wide analytics and AI/ML capabilities. This role will be instrumental in designing, developing, and optimising robust data pipelines within an AWS ecosystem.

As a Data Engineer, you will be responsible for developing high-performance ETL/ELT pipelines, processing large-scale structured and unstructured data, and delivering reliable, secure, and cost-efficient data flows to support data lakes, data warehouses, and feature stores. You’ll work closely with cross-functional teams to drive innovation and enable data-driven decision-making.

 

What you’ll do:

• Design, build, and maintain automated data pipelines using AWS Glue, Lambda, EMR, Kinesis, and Step Functions
• Ingest and transform data at scale, supporting both batch and real-time processing
• Implement CDC mechanisms for real-time data streaming and synchronisation
• Ensure structured data flows into data lakes (S3), Redshift, and analytical platforms
• Enable AI/ML teams by delivering high-quality, ready-to-use datasets
• Collaborate with engineering teams to provide automated, self-service data access
• Optimise pipeline performance to reduce latency and cost
• Build CI/CD pipelines for deployment, testing, and monitoring of workflows
• Use Terraform and CloudFormation for Infrastructure as Code (IaC) deployment
• Implement monitoring, version control, rollback strategies, and alerting (AWS CloudWatch)
• Stay current with emerging cloud-based data engineering trends and best practices

Your expertise:

• 4–5 years of experience in cloud-based data engineering
• Strong programming skills (Python, Scala, Java, or C++)
• Proficiency with AWS services: Glue, Redshift, S3, Lambda, EMR, IAM, RDS, and Athena
• Experience with distributed data processing tools (Apache Spark, AWS Glue)
• Familiarity with streaming tools like Kafka or Kinesis, and CDC-based workflows
• Strong understanding of data modelling, APIs, and event-driven architectures
• Experience with CI/CD, Git, and Infrastructure as Code (Terraform, CloudFormation)
• AWS certifications (Data Analytics or Solutions Architect) are advantageous

Soft skills competencies

• Technical Acumen: Stays up to date with evolving data platforms, tools, and security
• Analytical Thinking: Solves root causes, connects insights from various data sources
• Results Orientation: Focuses on delivery, performance, and measurable outcomes
• Communication: Clear, structured communicator in both written and verbal formats
• Collaboration: Works effectively across teams, building trusted partnerships
• Adaptability: Thrives in dynamic environments and adopts new tools quickly

Other information applicable to the opportunity:

• Permanent Position
• Location: Cape Town - Hybrid (4 days per week)