Data Engineer II

Recruiter:

CSG Recruit

Job Ref:

SORDEng

Date posted:

Wednesday, May 19, 2021

Location:

CapeTown, South Africa

Salary:

750000

SUMMARY:
Data Engineer II

POSITION INFO:

Roles/Responsibilities:

Design and develop data feeds from an on-premise environment into a datalake environment in an AWS cloud environment
Design and develop programmatic transformations of the solution, by correctly partitioning, formatting and validating the data quality
Design and develop programmatic transformation, combinations and calculations to populate complex datamarts based on feed from the datalake
Provide operational support to datamart datafeeds and datamarts
Design infrastructure required to develop and operate datalake data feeds
Design infrastructure required to develop and operate datamarts, their user interfaces and the feeds required to populate the datalake.

Job Related Experience required:

Business Intelligence (3-5 years)
Extract Transform and Load (ETL) processes (3-5 years)
Agile exposure, Kanban or Scrum

Job Related Knowledge required:

Creating data feeds from on-premise to AWS Cloud (1 year)
Support data feeds in production on break fix basis (1 year)
Creating data marts using Talend or similar ETL development tool (2 years)
Manipulating data using python and pyspark (1 year)
Processing data using the Hadoop paradigm particularly using EMR, AWS’s distribution of Hadoop (1 year)
Devop for Big Data and Business Intelligence including automated testing and deployment (1 year)

Skills Required:

Talend
AWS: EMR, EC2, S3
Python
PySpark or Spark
BI Data Modelling
SQL

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

AWS Data Engineer
Location: Cape Town
Salary:
1 day ago

Senior Data Engineer
Location: Cape Town
Salary: Market Related Monthly
Job DescriptionSummaryOur client is seeking a highly experienced Senior data engineer to lead the design, development, and optimization of data pipelines, APIs, and data platforms. This role will focus on ETL/ELT processes using Matillion and Snowflake, API development, and integration with machine learning workflows and databricks. The ideal candidate will have a strong background in data e...
2 days ago

Senior Data Engineer - Contract - Onsite
Location: Cape Town
Salary: R600 - 695 per hour
Senior data engineer - Contract - Onsite
2 days ago

Data Engineer
Location: Cape Town
Salary: Annually
Love Making data Flow? Our clients are constantly looking for talented data engineers to architect, build, and maintain scalable data solutions. Work alongside data-driven teams to design efficient pipelines, ensure data integrity, and empower businesses with actionable insights.
7 days ago

Data Engineer / Technical Generalist
Location: Cape Town
Salary:
11 days ago

Data Engineer – Cape Town – up to R500k per annum
Location: Capetown
Salary: 500000
data engineer – Cape Town – up to R500k per annum
21 days ago

Data Engineer (Azure Data Factory & Databricks)
Location: Cape Town
Salary:
30 days ago

Data Center Electrical Engineer
Location: Cape Town
Salary:
37 days ago

Data Engineer
Location: Cape Town
Salary:
41 days ago

Data Center Genset Engineer
Location: Cape Town
Salary:
41 days ago

Data Engineer II

Similar jobs you might be interested in:

Create a free job alert for Data Engineer II in CapeTown