Data Engineer III

Recruiter:

CSG Recruit

Job Ref:

SORDEng2

Date posted:

Wednesday, May 19, 2021

Location:

CapeTown, South Africa

Salary:

1300000

SUMMARY:
Data Engineer III

POSITION INFO:

Roles / Responsibilities:

Design and develop data feeds from an on-premise environment into a datalake environment in an AWS cloud environment
Design and develop programmatic transformations of the solution, by correctly partitioning, formatting and validating the data quality
Design and develop programmatic transformation, combinations and calculations to populate complex datamarts based on feed from the datalake
Provide operational support to datamart datafeeds and datamarts
Design infrastructure required to develop and operate datalake data feeds
Design infrastructure required to develop and operate datamarts, their user interfaces and the feeds required to populate the datalake.

JOB RELATED EXPERIENCE

Business Intelligence (5+ years)
Extract Transform and Load (ETL) processes (5+ years)
Cloud AWS (2+ years)
Agile exposure, Kanban or Scrum (2+ years)
Planning and team leading skills
Strong focus on data engineering, i.e. building pipelines, big data environments

JOB RELATED KNOWLEDGE

Creating data feeds from on-premise to AWS Cloud (2 years)
Support data feeds in production on break fix basis (2 years)
Creating data marts using Talend or similar ETL development tool (4 years)
Manipulating data using python and pyspark (2 years)
Processing data using the Hadoop paradigm particularly using EMR, AWS’s distribution of Hadoop (2 years)
Devop for Big Data and Business Intelligence including automated testing and deployment (2 years)

JOB RELATED SKILLS

Talend
AWS: EMR, EC2, S3
Python
PySpark or Spark
BI Data Modelling
SQL

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Data Engineer - AZURE
Location: Cape Town
Salary: 60 000 Monthly
the client is seeking an Azure data engineer who will be a cornerstone in constructing the backbone of our cloud-based fitness experiences. Your expertise will not only involve building and maintaining robust data solutions but also ensuring that these systems are insightful and interactive, utilizing Azure data Factory, databricks, and Power BI to transform data into action.
13 days ago

DevOps Data Engineer
Location: Cape Town
Salary:
14 days ago

Data Engineer
Location: Cape Town
Salary:
Requirements:Degree in BSc computer science, B.IT or informatics, BSc in Actuarial science or related degreeAsset Management experience ( Advantageous) Experience in Information & technology services Experience in data integration & transformation
24 days ago

DATA ENGINEER - AGRI-TECH (STELLENBOSCH)
Location: Stellenbosch
Salary:
Calling all QA warriors, code crusaders, and glitch gladiators!Our client, isn't just about growing crops, they're growing a revolution in Agri-Tech. And they need a superpowered Software Tester like you to join the fight against buggy software and ensure their tech is as fresh as their produce.Forget boring spreadsheets and endless tickets. This is a mission for those who thrive on excitement, im...
26 days ago

Emergency Response Expert
Location: Capetown
Salary:
Emergency Response Expert
3 days ago

Project Engineer - Order Processing (Mechanical)
Location: Cape Town
Salary: 450 00
Our client, a leading global manufacturer specializing in resource-saving evaporative cooling equipment, is looking for an experienced Order Process engineer to lead their team in Cape Town. The role entails overseeing the accurate and timely processing of all orders, particularly focusing on large and complex orders. As the liaison between field representatives, engineering, and manufacturin...
4 days ago

BI Engineer
Location: Cape Town
Salary:
Are you passionate about transforming data into actionable insights? Do you have a knack for designing and implementing robust business intelligence solutions?
7 days ago

BI Engineer
Location: Cape Town
Salary:
Are you passionate about transforming data into actionable insights? Do you have a knack for designing and implementing robust business intelligence solutions?
10 days ago

Senior Execution Framework Engineer
Location: Cape Town
Salary:
The Senior Execution Framework engineer design, prototype, implement and integrate systems that bring state-of-the-art algorithms to real-world scientific processing pipelines for the telescope. Telescope :processing functions" / "execution engine" / interfaces are key architectural features that are meant to not only enable scalability but also act as common ground for the integration of new and ...
10 days ago

Senior Pipeline Engineer
Location: Cape Town
Salary:
The Senior Pipeline engineer will provide subject matter expertise to the Computing and Software Telescope Operations team in the analysis, development, operation and maintenance of the telescope astronomy pipelines and other data analysis products such as the QA matrices. They're responsible for the identification, research and analysis of the problem sets, investigation of potential solutions, i...
10 days ago

Data Engineer III

Similar jobs you might be interested in:

Create a free job alert for Data Engineer III in CapeTown