2022_51 Senior Data Engineer

Recruiter:

Data Centrix

Job Ref:

JHB005940/Shadi

Date posted:

Tuesday, June 7, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
Datacentrix Client based in Johannesburg is looking for a Senior Data Engineer to enable the delivery of business processes, communication, collaboration and knowledge management through the deployment, support and maintenance of enterprise IT/OT technology, infrastructure, applications and Mine Technical systems.

JOB DESCRIPTION:

Formal qualifications:
An undergraduate qualification (Bachelor's degree or equivalent) in the relevant IM discipline and/or Technical competencies and certification with relevant years of experience in a similar role.

Role-specific knowledge:

Data Lake
Data Modeling
Data Architecture
Azure Data Environment

Specialist Areas:
Strong experience of building large scale file shipping and pipelines, ideally using Azure services such as AzCopy and Azure Data Lake.
Experience of managing unstructured file meta-data, conversions, standardisation and related workflows.
Experience of building analysis jobs that scale on technologies such as Databricks or Azure Batch.

Key Skills:

Phyton – Proficient
PySpark - Proficient
SQL – Competent
Solution Architecture – Competent
API Design - Competent
Containers – Competent
CI/CD – Competent
Azure Cloud - Competent
Data Stream patterns and technology – Proficient
Data stream patterns and technology – Proficient
Data engineering design patterns – Competent
Mining data – Beneficialy

Responsibilities

Develop ETL pipelines. The data transformations will be developed in Azure Databricks using Python and on Azure SQL using T-SQL and deployed using ARM templates
Combine and curate data in a central data lake
Serve data for application and analytics through a variety of technologies such as SQL, Server Synapse, CosmosDB and TSI
Build transformation pipelines into dimensions and facts and therefore a strong knowledge of standard BI concepts is mandatory
Build stream pipelines leverage IoT Hub, Event Hub, Databricks streaming and other Azure stream technologies.
Work in a fluid environment with changing requirements whilst maintaining absolute attention to detail.

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Financial Manager
Location: Johannesburg
Salary: 60000 Monthly
Shape the financial future of a global leader. Join a large-scale global company with operations in South Africa and the USA, employing over professionals across multiple regions.
Today

Executive Manager: Digital Services
Location: Pretoria
Salary:
Executive Manager: Digital ServicesThis role will serve as the delegated Information Officer, ensuring compliance with applicable data privacy legislation while providing strategic leadership in digital transformation. The incumbent will integrate digital knowledge, technologies, and capabilities into policies, architectures, and operational practices to enable the organisation to manage informati...
Today

Senior Growth Marketer
Location: Johannesburg
Salary:
Today

Product Sales Manager | Comminution & Process Technology
Location: Johannesburg
Salary: Market-related
Lead the strategic growth of a Comminution and Process Equipment division by managing complex tender cycles and high-value capital equipment sales.
Today

SAP Basis Consultants
Location: Centurion
Salary:
A senior SAP Basis Consultant responsible for the end-to-end management of complex SAP landscapes, spanning on-premises and cloud solutions. The role includes leading S/4HANA conversions, Unicode transformations, BW platforms, Solution Manager operations, and SAP Analytics Cloud (SAC) integrations, ensuring high availability, performance, and seamless data flow across the ecosystem.
Today

Revenue Controller
Location: Johannesburg
Salary: 450 000 Annually
Ready to take ownership of revenue in a fast-moving, tech-driven environment? Our client, an innovative company within the IT industry, is looking for a detail-oriented and analytical Revenue Controller to join their team in Johannesburg East.
Today

Part-Time Accountant | Remote (South Africa) | Permanent (25 hours per week)
Location: Johannesburg
Salary:
Today

Head of Learning and Development
Location: Johannesburg
Salary: Market Related
Head of Learning and Development
Today

Senior Conveyancing Secretary
Location: Bedfordview
Salary:
Today

Executive Head: Software Development
Location: Bedfordview, Gauteng
Salary: R1,5 m – R1.8 per annum
Executive Head: Software Development
Today

2022_51 Senior Data Engineer

Similar jobs you might be interested in:

Create a free job alert for 2022_51 Senior Data Engineer in Johannesburg