Senior Data Engineer (CH1223)

SUMMARY:
-

POSITION INFO:

Our client, is a leading South African payment aggregator, enabling secure and efficient bill payments across a wide network of payment channels. They process millions of transactions monthly and are building a modern data infrastructure to support financial reporting, fraud detection, AI, and operational analytics.

They are seeking a skilled Senior Data Engineer to join our growing team. You will be instrumental in designing, building, and maintaining scalable, robust data pipelines and warehouse solutions that integrate transactional, accounting, and CRM data across multiple systems including SQL Server, PostgreSQL, Sage One, Xero, and Dataverse.

• Responsible for the preparation of data products and advanced technical thought leadership for the organisation.
• Deliver fit-for-purpose data products for consumption by advanced analytics.
• Provide clean, usable data to whomever may require it.
• Design, construct, install, test and maintain highly scalable data management systems.
• Build high-performance prototypes and proof of concepts. Research opportunities for data acquisition and new uses for existing data.
• Develop data set processes for data modelling, mining and production.
• Integrate new data management technologies and software engineering tools into existing structures and create custom software components (e.g. specialised UDFs) and analytics applications.
• Employ a variety of languages and tools (e.g. scripting languages) to marry systems together.
• Install and update disaster recovery procedures.
• Collaborate with data architects, modelers and IT team members on project goals and recommend ways to improve data reliability, efficiency and quality.
• In conjunction with Data Architect, determine which data management systems are appropriate.

Key responsibilities:

Data Pipeline Development:

• Build and maintain robust ETL/ELT pipelines using Python, SQL, and cloud native tools to ingest and transform data from 40+ payment networks and internal systems.

Data Integration:

• Join and harmonize data across diverse sources including transactional databases, accounting systems, and CRM platforms.

Collaboration & Governance:

• Work closely with developers, analysts, and business stakeholders to ensure data accuracy, security, and accessibility.

Monitoring & Optimization:

• Implement data quality checks, performance tuning, and monitoring for pipelines handling tens of millions of rows monthly.

Documentation & Standards:

• Maintain clear documentation and follow best practices for data architecture, version control, and deployment.

Tech Stack:

Databases & Warehousing: SQL server, PostgreSQL

Languages: Python, SQL

Essential minimum qualifications, skills, and experience:

• Degree in a quantitative field (e.g. Information Systems, Computer Science, Mathematics, Statistics, or Finance).
• 8+ years relevant experience in data engineering or related roles.
• Proven experience in Python-based data engineering projects
• Strong proficiency in SQL and Python for data transformation and automation.
• Experience with data orchestration tools (e.g., Airflow).
• Excellent problem-solving and communication skills.

Advantageous Skills & Experience:

• Understanding of dimensional modelling and data warehouse design.
• Experience with systems like SQL Server, PostgreSQL, Sage One, Xero, Business.
• Central, HubSpot and Microsoft Dataverse is a plus.
• Agile (Scrum) development experience.
• Experience in the payments or financial services industry.

General:

• We are committed to fair and inclusive hiring. All suitably qualified applicants are welcome to apply.
• Preference may be given in line with our client’s Employment Equity plan and applicable legislation.
• Applicants must have the legal right to work in South Africa at the time of application.
• Only shortlisted candidates will be contacted. If you do not hear from us within 30 days, please consider your application unsuccessful.
• Please include your current remuneration (CTC), salary expectation, and notice period (optional but helpful for screening).
• By applying, you consent to the processing of your personal information for recruitment purposes in accordance with POPIA.