Data Engineer

Recruiter:

Network Recruitment International

Job Ref:

NCS000170/NM

Date posted:

Thursday, June 30, 2022

Location:

Johannesburg, South Africa

Salary:

R620 Hourly

JOB SUMMARY:
A technology focussed individual, data engineers will need to design, configure, develop and deploy data transformations.

The purpose of Global Information Management (IM) is to enable the delivery of business processes, communication, collaboration and knowledge management through the deployment, support and maintenance of enterprise IT/OT technology, infrastructure, applications and Mine Technical systems.

JOB DESCRIPTION:

Develop ETL pipelines. The data transformations will be developed in Azure Databricks using Python and on Azure SQL using T-SQL and deployed using ARM templates
Combine and curate data in a central data lake
Serve data for application and analytics through a variety of technologies such as SQL, Server Synapse, CosmosDB and TSI
Build transformation pipelines into dimensions and facts and therefore a strong knowledge of standard BI concepts is mandatory
Build stream pipelines leverage IoT Hub, Event Hub, Databricks streaming and other Azure stream technologies.
Work in a fluid environment with changing requirements whilst maintaining absolute attention to detail.

This work involves analysis, reflection and the use of data over a more extended period, to anticipate problems. It requires knowledge of the overall system and focuses on the direct application and improvement of systems

Key Skills:
· Phyton – Proficient
· PySpark - Proficient
· SQL – Competent
· Solution Architecture – Competent
· API Design - Competent
· Containers – Competent
· CI/CD – Competent
· Azure Cloud - Competent
· Data Stream patterns and technology – Proficient
· Data stream patterns and technology – Proficient
· Data engineering design patterns – Competent
· Mining data – Beneficial

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Azure Data Engineer
Location: Johannesburg
Salary: negotiable
Our client based in Sandton is looking for an Azure data engineer to join their team. 12-month contract is renewable.
1 day ago

Junior Data Engineer
Location: Sandton
Salary: R25k - 30k per month
Building, maintaining, and optimising data pipelines and infrastructure, enabling advanced analytics
9 days ago

Data Engineer
Location: Johannesburg
Salary: Monthly
Psybergate is an IT company that builds bespoke software solutions and provides highly skilled resources to its clients.We are looking for a data engineer to join our financial services client based in Sandton on a contract basis.
10 days ago

Data Engineer
Location: Sandton
Salary: R75k - 100k per month
Building, maintaining, and optimising data pipelines and infrastructure, enabling advanced analytics
10 days ago

Data Engineer (AI & Machine Learning)
Location: South Africa
Salary:
This Easter, we are on the hunt for a data engineer with AI & Machine Learning experience, and yes, this is the kind of opportunity worth putting in your basket. If you love combining data engineering, cloud platforms and ML operations in a modern, fast paced environment this could be your golden egg.
15 days ago

Azure Data Engineer (Mid to Senior)
Location: Johannesburg
Salary: market related
This role focuses on designing, building, and supporting data solutions using Azure technologies and Power BI.
63 days ago

Informatica Data Engineer
Location: Johannesburg
Salary:
63 days ago

Senior Enterprise Data Engineer
Location: Johannesburg
Salary:
65 days ago

Senior Data Engineer
Location: Johannesburg
Salary:
65 days ago

BI Data Engineer
Location: Johannesburg
Salary:
As a Senior data engineer in Business Intelligence, you’ll be the architect behind scalable, secure data systems that power real-time insights across the logistics value chain. From building high-performance ELT pipelines and integrating complex platforms to delivering clean, trusted datasets, you’ll enable automation, cost optimisation, and smarter decision-making. This role is all ab...
3 days ago

Data Engineer

Similar jobs you might be interested in:

Create a free job alert for Data Engineer in Johannesburg