Senior Data Engineering Technical Lead

SUMMARY:
Lead with Impact. Engineer the Future of Data.
Step into a role where scale is not only theoretical, but real. Our client, a top-tier technology consultancy driven to deliver meaningful, high-impact results, is on the hunt for Senior Data Engineer who is passionate about tackling complex data challenges, leading technical initiatives, and building systems that operate at multi-petabyte scale. This role offers the ideal candidate the opportunity to combine, shaping high-impact data solutions while mentoring others and driving innovation from concept to production.

POSITION INFO:
Key Duties and Responsibilities Build & Scale Data Systems Architect and deliver large-scale data migration strategies for identity transformation Build robust batch and streaming pipelines using Apache Spark Develop data lineage, validation, and reconciliation frameworks to ensure data integrity Orchestrate highly parallel data workflows for performance and reliability Lead upgrades and optimisation of Spark and JVM environments in production Implement monitoring and observability to ensure system health and performance Design test frameworks to validate complex migration scenarios Lead & Elevate Engineering Teams Take ownership of critical components and drive engineering excellence Mentor team members through pair programming, reviews, and coaching Lead technical discussions and influence architecture decisions Proactively identify risks and remove delivery blockers Collaborate across teams to align technical solutions with business goals Technical Expertise: Your Tech Stack Languages: Scala, Java (Maven\/Gradle) Big Data: Apache Spark, HDFS, S3, GCS Cloud: AWS, Azure, GCP, Databricks Containers: Kubernetes (EKS) Streaming: Kafka, Kinesis (bonus) Data Formats: Iceberg, Parquet, ORC, Avro (bonus) Observability: Prometheus, Grafana, Spark UI (bonus) Essential Skills, Experience, and Qualifications 8+ years' experience in data engineering or software engineering Strong expertise in large-scale data migration and ETL pipelines Deep understanding of distributed systems and scalable architecture Hands-on experience with cloud data platforms and containerised environments Knowledge of data security and privacy regulations (e.g., GDPR) A passion for building reliable, high-performance systems Personal Attributes A natural leader and hands-on builder Strong problem-solver who thrives on complexity Collaborative mindset with a focus on knowledge sharing Comfortable challenging ideas and contributing to better outcomes Committed to quality, ownership, and continuous improvement Why This Role? Work on cutting-edge, petabyte-scale data projects Influence high-value solutions across enterprise environments Grow through strong mentorship and leadership opportunities Be part of a culture built on trust, collaboration, and continuous learning Enjoy flexible working options and competitive rewards Hybrid- 2 days in-office. If you are excited by big data, complex systems, and meaningful impact, this is your opportunity to take your career to the next level. Apply Now. For more IT jobs, please visit I also specialise in recruiting in the following: Software Development Data Engineer Data Analyst Infrastructure Architecture ... and more! If you have not had any response in two weeks, please consider the vacancy application unsuccessful. Your profile will be kept on our database for any other suitable roles \/ positions. For more information contact: Evandria Rademeyer Researcher