Senior Data Engineer

SUMMARY:
-

POSITION INFO:

About the Role
We are seeking a highly skilled and hands-on Senior Data Engineer to lead the design, development, and optimization of scalable data pipelines. You will play a critical role in architecting and maintaining the data infrastructure that supports enterprise analytics, business intelligence, and operational systems.

This position requires deep technical expertise in data integration, strong data architecture capabilities, and the ability to thrive in high-volume, business-critical data environments where performance, reliability, and scalability are essential.

Key Responsibilities

1. Pipeline Development & Data Integration
   Design, develop, and maintain robust ETL/ELT pipelines using Qlik Talend to process large datasets from diverse source systems
   • Implement reusable, scalable integration patterns across structured and semi-structured data sources
   • Ensure pipeline reliability through monitoring, logging, and automated validation mechanisms
   • Manage end-to-end data lifecycle from ingestion through transformation and delivery
2. Data Replication & Real-Time Processing
   Implement and manage log-based Change Data Capture (CDC) and near-real-time ingestion using Qlik Replicate
   • Support low-latency data availability for analytics and operational reporting
   • Oversee database replication strategies across heterogeneous systems
   • Ensure data consistency, reconciliation, and minimal downtime during migrations or platform upgrades
3. Data Architecture & Modeling
   Design and optimize scalable data models including Star Schema, Snowflake Schema, and Data Vault
   • Apply dimensional modeling principles and Kimball methodologies for analytics-ready data structures
   • Balance performance, storage efficiency, and maintainability in enterprise data warehouse environments
   • Contribute to architectural decisions regarding data platform evolution
4. Cloud & On-Premise Data Solutions
   Architect and manage hybrid data solutions across cloud platforms (AWS, Azure, GCP) and enterprise on-premise systems
   • Implement integrations with cloud data warehouses such as Snowflake, Redshift, BigQuery, or Synapse
   • Manage cloud storage solutions (S3, ADLS, GCS) for scalable data ingestion and processing
5. Performance Optimization & Governance
   Optimize complex SQL queries, transformations, and pipeline configurations for maximum efficiency
   • Conduct performance tuning across databases including PostgreSQL, SQL Server, and Oracle
   • Implement best practices for data quality, validation, and governance
   • Maintain documentation for pipelines, data models, and engineering standards
6. Collaboration & Mentorship
   Collaborate closely with Data Analysts, BI Developers, Data Scientists, and business stakeholders to translate requirements into scalable data solutions
   • Mentor junior data engineers and conduct structured code reviews
   • Contribute to the development of data engineering standards, CI/CD practices, and documentation frameworks
   
   

Requirements & Qualifications

Must-Have Skills & Experience:
• 6+ years of professional experience in data engineering, data integration, or software engineering focused on data platforms
• Expert-level hands-on experience with Qlik Talend (Data Integration or Studio) for complex ETL/ELT orchestration
• Proven production experience with Qlik Replicate for log-based CDC and database replication
• Advanced SQL proficiency for complex querying, transformations, and performance optimization
• Strong understanding of data modeling concepts including dimensional modeling and 3NF
• Experience designing analytics-ready data structures using Kimball methodologies

Highly Desirable Skills:
• Experience with cloud data warehouses (Snowflake, Redshift, BigQuery, Synapse)
• Experience with cloud storage solutions (S3, ADLS, GCS)
• Proficiency in Python or Scala for building custom transformations or API integrations
• Familiarity with orchestration tools such as Apache Airflow, Prefect, or cloud-native schedulers
• Exposure to big data frameworks such as Spark or Databricks
• Familiarity with streaming technologies such as Kafka
• Experience with version control (Git) and CI/CD practices for data pipeline deployment

Education

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or related technical field

Additional Information

• Employment Type: Full-time
• Salary: Market - Related / Negotiable
• Location Options: Johannesburg / Cape Town / Remote
• Must be located in South Africa or have a valid work permit for South Africa