Data Engineer

SUMMARY:
Location: Remote/Hybrid, based in Johannesburg
Position Type: Full-Time
Are you a skilled Data Engineer passionate about building scalable data solutions for cutting-edge AI and machine learning projects?
We are looking for a dynamic professional to contribute to creating  data infrastructures that enable innovation in AI, generative AI, and MLOps.

POSITION INFO:

We are seeking a skilled Data Engineer to design and optimize scalable data infrastructure for AI, machine learning, and generative AI initiatives. This role involves building ETL/ELT pipelines, managing AWS-based data systems, and collaborating with cross-functional teams to develop cutting-edge solutions. Ideal candidates will have experience in AWS services, MLOps frameworks, and tools like Airbyte, with a strong focus on automation, data governance, and performance optimization. If you’re passionate about leveraging data to drive innovation, we’d love to hear from you!

Key Responsibilities

• Data Pipeline Development: Design and maintain ETL/ELT pipelines using tools like Airbyte for seamless data integration.
• AWS Data Solutions: Build and manage data infrastructures with AWS services like S3, Redshift, Glue, Lambda, and Athena.
• Data Architecture: Develop scalable, cost-effective data systems for AI applications.
• AI Integration: Collaborate with data scientists and machine learning engineers to support AI and generative AI initiatives.
• MLOps Support: Implement and monitor machine learning models in production.
• Automation: Streamline workflows for data ingestion and feature engineering.
• Collaboration: Work with cross-functional teams to meet diverse data needs.
• Data Governance: Maintain data quality, compliance, and security.
• Troubleshooting: Ensure high availability and reliability of data systems.

Required Skills and Qualifications

• Education: Bachelor’s in Computer Science, Data Engineering, or a related field (Master’s preferred).
• Experience:
  • 3+ years in data engineering, focusing on AWS ecosystems.
  • Proficiency in building ETL/ELT pipelines with tools like Airbyte.
  • Expertise in MLOps frameworks and production-ready machine learning pipelines.
  • Strong Python programming skills (Java or Scala experience is a plus).
  • Proficiency in SQL and relational databases like PostgreSQL, MySQL, or MS SQL.
  • Experience with distributed data processing (e.g., Apache Spark, Kafka).
• Technical Expertise:
  • Deep knowledge of AWS services (S3, Redshift, Glue, Lambda, Athena, etc.).
  • Experience with modern data stack tools (e.g., DBT, Airflow).
  • Familiarity with data workflows for AI/ML, including generative AI models.
• Soft Skills:
  • Strong problem-solving skills for complex data challenges.
  • Excellent communication skills for collaboration with diverse teams.

Preferred Qualifications

• Experience with NoSQL databases (e.g., DynamoDB, MongoDB).
• Familiarity with containerization (Docker) and orchestration (Kubernetes).
• Understanding of LLMs, transformer architectures, and generative AI.
• Knowledge of BI tools like Tableau or Power BI.
• Experience with infrastructure-as-code tools like Terraform.

NB! This job is now closed. You can apply for other jobs by uploading your CV.