2022_51 Senior Data Engineer

Recruiter:

Data Centrix

Job Ref:

JHB005940/Shadi

Date posted:

Tuesday, June 7, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
Datacentrix Client based in Johannesburg is looking for a Senior Data Engineer to enable the delivery of business processes, communication, collaboration and knowledge management through the deployment, support and maintenance of enterprise IT/OT technology, infrastructure, applications and Mine Technical systems.

JOB DESCRIPTION:

Formal qualifications:
An undergraduate qualification (Bachelor's degree or equivalent) in the relevant IM discipline and/or Technical competencies and certification with relevant years of experience in a similar role.

Role-specific knowledge:

Data Lake
Data Modeling
Data Architecture
Azure Data Environment

Specialist Areas:
Strong experience of building large scale file shipping and pipelines, ideally using Azure services such as AzCopy and Azure Data Lake.
Experience of managing unstructured file meta-data, conversions, standardisation and related workflows.
Experience of building analysis jobs that scale on technologies such as Databricks or Azure Batch.

Key Skills:

Phyton – Proficient
PySpark - Proficient
SQL – Competent
Solution Architecture – Competent
API Design - Competent
Containers – Competent
CI/CD – Competent
Azure Cloud - Competent
Data Stream patterns and technology – Proficient
Data stream patterns and technology – Proficient
Data engineering design patterns – Competent
Mining data – Beneficialy

Responsibilities

Develop ETL pipelines. The data transformations will be developed in Azure Databricks using Python and on Azure SQL using T-SQL and deployed using ARM templates
Combine and curate data in a central data lake
Serve data for application and analytics through a variety of technologies such as SQL, Server Synapse, CosmosDB and TSI
Build transformation pipelines into dimensions and facts and therefore a strong knowledge of standard BI concepts is mandatory
Build stream pipelines leverage IoT Hub, Event Hub, Databricks streaming and other Azure stream technologies.
Work in a fluid environment with changing requirements whilst maintaining absolute attention to detail.

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

IT INFRASTRUCTURE AND OPERATIONS MANAGER
Location: Pretoria
Salary: Market related
This person is responsible for maintaining and managing an organization's technology infrastructure.
Today

Credit Controller
Location: Pretoria
Salary:
A rapidly growing Renewable Energy company, focused on delivering reliable solar and backup power solutions, is on the lookout for an experienced Credit Controller to maintain positive relationships with clients and internal stakeholders, while also safeguarding the financial health of the organisation.Pretoria | Permanent | R 12 000 pm
1 day ago

Senior Data Science Consultant
Location: South Africa
Salary: 1 Monthly
senior data Science Consultant
1 day ago

Technical Bid Coordinator
Location: Pretoria
Salary: Market related
The Technical Bid Coordinator will be responsible for managing the end-to-end bid and tender process
1 day ago

Senior Credit Controller
Location: Johannesburg
Salary: 430000 Annually
Are you passionate about credit control and financial excellence?If managing complex accounts, reducing risk, and keeping cash flow healthy is what drives you, then this senior Credit Controller opportunity in the FMCG industry could be your next big career move.
1 day ago

Hotspot Product and Project Manager
Location: Midrand
Salary: 45000 Monthly
Are you passionate about Wi-Fi performance, data-driven testing, and continuous improvement? Join a tech-driven team where your analytical mindset and problem-solving skills will help shape the future of hotspot connectivity in underserved communities.
1 day ago

PHP Developer (Back-End)
Location: Johannesburg
Salary: 35000 Monthly
Are you a passionate PHP Developer ready to take your back-end skills to the next level? Join a fast-paced, tech-driven team where innovation and collaboration are at the heart of everything we do.
1 day ago

Financial Accountant
Location: Johannesburg
Salary:
Step into a dynamic manufacturing environment where numbers tell stories, insights drive action, and your expertise as a CA(SA) shapes strategic growth. If you’re a data-driven finance professional who thrives in a fast-paced multinational setting, this opportunity was made for you
1 day ago

Predictive Risk Storyteller
Location: Johannesburg
Salary:
Predictive Risk StorytellerDo you live for the thrill of data-driven decision-making? If your idea of fun is turning complex datasets into powerful, predictive scorecards that drive real business results — this one’s for you! We’re looking for an Predictive Risk Storyteller who doesn’t just build models but builds impact. Someone who thrives in fast-paced, innovative enviro...
1 day ago

General Manager
Location: Alberton
Salary: 20000
A Spanish-inspired restaurant known for its lively ambiance, tapas-style dining, and exceptional service is looking for a brilliant ððð¡ðð¥ðð ð ðð¡ðððð¥
1 day ago

2022_51 Senior Data Engineer

Similar jobs you might be interested in:

Create a free job alert for 2022_51 Senior Data Engineer in Johannesburg