2022_51 Senior Data Engineer

Recruiter:

Data Centrix

Job Ref:

JHB005940/Shadi

Date posted:

Tuesday, June 7, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
Datacentrix Client based in Johannesburg is looking for a Senior Data Engineer to enable the delivery of business processes, communication, collaboration and knowledge management through the deployment, support and maintenance of enterprise IT/OT technology, infrastructure, applications and Mine Technical systems.

JOB DESCRIPTION:

Formal qualifications:
An undergraduate qualification (Bachelor's degree or equivalent) in the relevant IM discipline and/or Technical competencies and certification with relevant years of experience in a similar role.

Role-specific knowledge:

Data Lake
Data Modeling
Data Architecture
Azure Data Environment

Specialist Areas:
Strong experience of building large scale file shipping and pipelines, ideally using Azure services such as AzCopy and Azure Data Lake.
Experience of managing unstructured file meta-data, conversions, standardisation and related workflows.
Experience of building analysis jobs that scale on technologies such as Databricks or Azure Batch.

Key Skills:

Phyton – Proficient
PySpark - Proficient
SQL – Competent
Solution Architecture – Competent
API Design - Competent
Containers – Competent
CI/CD – Competent
Azure Cloud - Competent
Data Stream patterns and technology – Proficient
Data stream patterns and technology – Proficient
Data engineering design patterns – Competent
Mining data – Beneficialy

Responsibilities

Develop ETL pipelines. The data transformations will be developed in Azure Databricks using Python and on Azure SQL using T-SQL and deployed using ARM templates
Combine and curate data in a central data lake
Serve data for application and analytics through a variety of technologies such as SQL, Server Synapse, CosmosDB and TSI
Build transformation pipelines into dimensions and facts and therefore a strong knowledge of standard BI concepts is mandatory
Build stream pipelines leverage IoT Hub, Event Hub, Databricks streaming and other Azure stream technologies.
Work in a fluid environment with changing requirements whilst maintaining absolute attention to detail.

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Buying Manager (Expat)
Location: Johannesburg
Salary: Market related
Our client is seeking a Buying Manager who will be responsible for overseeing the purchasing and pro
Today

Senior ERP Consultant
Location: Johannesburg
Salary: Monthly
We are looking for interesting people to join a dynamic team of happy and talented humans. If you are looking for a company that provides more than just being a number and want to grow and be offered the opportunity to expand your knowledge, then this is for you.
Today

Intermediate Project Manager
Location: Johannesburg
Salary: Market related
Intermediate Project Manager | For a forward-thinking company that specialises in utilizing geospatial data and technologies to help clients enhance their operational efficiencies and increase profitability.
Today

Wills Sales Manager
Location: Pretoria
Salary:
Today

Manager – Approval/Authorizations
Location: Johannesburg
Salary:
2 days ago

Store Manager
Location: Johannesburg
Salary:
Our client, an upmarket, fast-paced growing Brand of clothing is looking for a Store and Assistant Manager.
3 days ago

Team Leader & Training Coordinator
Location: Johannesburg
Salary: 55 000 Monthly
A well-established company located in Johannesburg North is looking for a Team Leader & Training Coordinator to join their team!
3 days ago

Stakeholder Engagement and Communications Specialist
Location: Johannesburg
Salary:
3 days ago

Business Analyst / Investment Reporting Specialist (CH1010)
Location: Pretoria
Salary:
3 days ago

SAP Developer (Banking 12 - month contract)
Location: Johannesburg
Salary:
3 days ago

2022_51 Senior Data Engineer

Similar jobs you might be interested in:

Create a free job alert for 2022_51 Senior Data Engineer in Johannesburg