Senior Data Engineer

SUMMARY:

POSITION INFO:
This isn't about maintaining legacy pipelines; you'll be leading high-stakes architectural design, specifically engineering for complex producer and consumer Redshift cluster environments on AWS. Our client operates at an incredible scale, running a powerhouse high-volume batch infrastructure while aggressively expanding our real-time footprint. Crucially, you'll help spearhead the shift toward adopting AI and agentic practices to build smarter, self-healing, and highly robust pipelines. Your design decisions will directly shape the data foundation for market-disrupting services and millions of banking clients. This is a high-autonomy role with deep technical ownership, and a chance to make a systemic impact. Purpose Statement To design, build, and optimise robust, scalable, and efficient data platforms and infrastructure that enable data-driven decision-making across the organisation. You will contribute to solution design, build and enhance data pipelines, and ensure the reliability, performance, and scalability of the data ecosystem. What you'll be doing: Design and build scalable data pipelines and data platforms (batch and streaming) Develop high-performance data solutions using Python and advanced SQL (Redshift focus) Contribute to solution design alongside Data Architects Build and maintain cloud-based data infrastructure (preferably AWS) Implement infrastructure as code (e.g., Terraform) and CI\/CD practices Work with Git, deployment pipelines, and SDLC processes Design and optimise data lake, data warehouse, and integration patterns Enable data quality, monitoring, and governance frameworks Mentor and guide junior engineers and contribute to team capability building Partner with stakeholders to translate business requirements into scalable data solutions What our client is looking for: Bachelor's Degree in Information Technology or related field or ideally an Honours Degree in IT, Computer Science, or related field Proven experience in data engineering at scale (senior\/staff level) Strong experience in: Python Advanced SQL (query optimisation, tuning, indexing - Redshift preferred) Cloud platforms (AWS preferred) Experience with: Data warehousing and data lake architectures CI\/CD, Git, and deployment practices Infrastructure as code (e.g., Terraform) Building and maintaining production-grade data pipelines Ability to work independently and contribute quickly in a complex environment Experience with: Azure or GCP (with proven ability to transition to AWS) Power BI and Microsoft data stack exposure Real-time \/ streaming data architectures Workflow orchestration tools Containerised solutions Exposure to machine learning data pipelines or predictive analytics environments This is not a people management role, but you will: Provide technical mentorship and guidance to junior team members Help define frameworks, standards, and best practices Contribute to scaling and improving the engineering capability of the team Our client is looking for someone who is: Self-managed and accountable Collaborative and team-oriented Comfortable working in a fast-paced, evolving environment Minimum Experience: A minimum of 8-10+ years' experience in data engineering or related fields (aligned to staff-level capability) Proven track record of: Designing and delivering large-scale data solutions Driving business impact through data platforms Mentoring and developing other engineers Education (Minimum) Bachelor's Degree in Information Technology Education (Ideal or Preferred) Honours Degree in Information Technology - Computer Science or Information Technology Knowledge and Experience Knowledge: Expert level SQL knowledge with query optimisation and tuning and indexing strategies. Architecture design for batch and streaming systems Deep knowledge of multiple programming languages and implementation of advanced algorithms for data processing. Design enterprise wide real-time \/ batch data integration architectures. Advanced knowledge of one cloud platforms data services. Design and implementation of data lake, data warehouse or other patterns. Implementation of enterprise-wide data governance frameworks with automated quality and monitoring. Ability to performance optimize across heterogeneous systems. Design of multi-environment GitOps strategies, that includes feature flags and other advanced techniques. Design & Implementation of containerized data solutions and infrastructure using infrastructure as code Design and configuration enterprise workflow orchestration tools and pipelines Experience: Length of experience required is conditional on the qualifications obtained but must include: Proven track record of leading large-scale data engineering projects and driving business impact through data solutions Experience with advanced data engineering tools and software (e.g., SQL, Python, Java, Apache Spark, Hadoop). Experience with machine learning algorithms and their application in predictive analytics Experience in designing and implementing complex data architecture and infrastructure. With cloud computing platforms (e.g., AWS, Azure, Google Cloud) and their data services 2. Experience in aligning data engineering projects with business strategy and goals. Experience in providing data-driven recommendations to senior management and influencing decision-making. Experience in leading and mentoring less experienced engineers and data teams. In people development, sharing and training. Should you not receive a response from us within one week of your application, your application has unfortunately not been successful. You can visit our website for more vacancies: \/ LinkedIn