Senior Data Engineering Technical Lead

SUMMARY:
Lead with Impact. Engineer the Future of Data.
Step into a role where scale is not only theoretical, but real. Our client, a top-tier technology consultancy driven to deliver meaningful, high-impact results, is on the hunt for Senior Data Engineer who is passionate about tackling complex data challenges, leading technical initiatives, and building systems that operate at multi-petabyte scale. This role offers the ideal candidate the opportunity to combine, shaping high-impact data solutions while mentoring others and driving innovation from concept to production.

POSITION INFO:
Key Duties and Responsibilities
Build & Scale Data Systems

• Architect and deliver large-scale data migration strategies for identity transformation
• Build robust batch and streaming pipelines using Apache Spark
• Develop data lineage, validation, and reconciliation frameworks to ensure data integrity
• Orchestrate highly parallel data workflows for performance and reliability
• Lead upgrades and optimisation of Spark and JVM environments in production
• Implement monitoring and observability to ensure system health and performance
• Design test frameworks to validate complex migration scenarios

• Take ownership of critical components and drive engineering excellence
• Mentor team members through pair programming, reviews, and coaching
• Lead technical discussions and influence architecture decisions
• Proactively identify risks and remove delivery blockers
• Collaborate across teams to align technical solutions with business goals

• Languages: Scala, Java (Maven/Gradle)
• Big Data: Apache Spark, HDFS, S3, GCS
• Cloud: AWS, Azure, GCP, Databricks
• Containers: Kubernetes (EKS)
• Streaming: Kafka, Kinesis (bonus)
• Data Formats: Iceberg, Parquet, ORC, Avro (bonus)
• Observability: Prometheus, Grafana, Spark UI (bonus)

• 8+ years’ experience in data engineering or software engineering
• Strong expertise in large-scale data migration and ETL pipelines
• Deep understanding of distributed systems and scalable architecture
• Hands-on experience with cloud data platforms and containerised environments
• Knowledge of data security and privacy regulations (e.g., GDPR)
• A passion for building reliable, high-performance systems

• A natural leader and hands-on builder
• Strong problem-solver who thrives on complexity
• Collaborative mindset with a focus on knowledge sharing
• Comfortable challenging ideas and contributing to better outcomes
• Committed to quality, ownership, and continuous improvement

• Work on cutting-edge, petabyte-scale data projects
• Influence high-value solutions across enterprise environments
• Grow through strong mentorship and leadership opportunities
• Be part of a culture built on trust, collaboration, and continuous learning
• Enjoy flexible working options and competitive rewards
• Hybrid- 2 days in-office.