Data Engineer

SUMMARY:
Location: Remote/Hybrid, based in Johannesburg
Position Type: Full-Time
Are you a skilled Data Engineer passionate about building scalable data solutions for cutting-edge AI and machine learning projects?
We are looking for a dynamic professional to contribute to creating  data infrastructures that enable innovation in AI, generative AI, and MLOps.

POSITION INFO:
We are seeking a skilled Data Engineer to design and optimize scalable data infrastructure for AI, machine learning, and generative AI initiatives. This role involves building ETL\/ELT pipelines, managing AWS-based data systems, and collaborating with cross-functional teams to develop cutting-edge solutions. Ideal candidates will have experience in AWS services, MLOps frameworks, and tools like Airbyte, with a strong focus on automation, data governance, and performance optimization. If you're passionate about leveraging data to drive innovation, we'd love to hear from you! Key Responsibilities Data Pipeline Development: Design and maintain ETL\/ELT pipelines using tools like Airbyte for seamless data integration. AWS Data Solutions: Build and manage data infrastructures with AWS services like S3, Redshift, Glue, Lambda, and Athena. Data Architecture: Develop scalable, cost-effective data systems for AI applications. AI Integration: Collaborate with data scientists and machine learning engineers to support AI and generative AI initiatives. MLOps Support: Implement and monitor machine learning models in production. Automation: Streamline workflows for data ingestion and feature engineering. Collaboration: Work with cross-functional teams to meet diverse data needs. Data Governance: Maintain data quality, compliance, and security. Troubleshooting: Ensure high availability and reliability of data systems. Required Skills and Qualifications Education: Bachelor's in Computer Science, Data Engineering, or a related field (Master's preferred). Experience: 3+ years in data engineering, focusing on AWS ecosystems. Proficiency in building ETL\/ELT pipelines with tools like Airbyte. Expertise in MLOps frameworks and production-ready machine learning pipelines. Strong Python programming skills (Java or Scala experience is a plus). Proficiency in SQL and relational databases like PostgreSQL, MySQL, or MS SQL. Experience with distributed data processing (e.g., Apache Spark, Kafka). Technical Expertise: Deep knowledge of AWS services (S3, Redshift, Glue, Lambda, Athena, etc.). Experience with modern data stack tools (e.g., DBT, Airflow). Familiarity with data workflows for AI\/ML, including generative AI models. Soft Skills: Strong problem-solving skills for complex data challenges. Excellent communication skills for collaboration with diverse teams. Preferred Qualifications Experience with NoSQL databases (e.g., DynamoDB, MongoDB). Familiarity with containerization (Docker) and orchestration (Kubernetes). Understanding of LLMs, transformer architectures, and generative AI. Knowledge of BI tools like Tableau or Power BI. Experience with infrastructure-as-code tools like Terraform.

NB! This job is now closed. You can apply for other jobs by uploading your CV.