2022_51 Senior Data Engineer

Recruiter:

Data Centrix

Job Ref:

JHB005940/Shadi

Date posted:

Tuesday, June 7, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
Datacentrix Client based in Johannesburg is looking for a Senior Data Engineer to enable the delivery of business processes, communication, collaboration and knowledge management through the deployment, support and maintenance of enterprise IT/OT technology, infrastructure, applications and Mine Technical systems.

JOB DESCRIPTION:

Formal qualifications:
An undergraduate qualification (Bachelor's degree or equivalent) in the relevant IM discipline and/or Technical competencies and certification with relevant years of experience in a similar role.

Role-specific knowledge:

Data Lake
Data Modeling
Data Architecture
Azure Data Environment

Specialist Areas:
Strong experience of building large scale file shipping and pipelines, ideally using Azure services such as AzCopy and Azure Data Lake.
Experience of managing unstructured file meta-data, conversions, standardisation and related workflows.
Experience of building analysis jobs that scale on technologies such as Databricks or Azure Batch.

Key Skills:

Phyton – Proficient
PySpark - Proficient
SQL – Competent
Solution Architecture – Competent
API Design - Competent
Containers – Competent
CI/CD – Competent
Azure Cloud - Competent
Data Stream patterns and technology – Proficient
Data stream patterns and technology – Proficient
Data engineering design patterns – Competent
Mining data – Beneficialy

Responsibilities

Develop ETL pipelines. The data transformations will be developed in Azure Databricks using Python and on Azure SQL using T-SQL and deployed using ARM templates
Combine and curate data in a central data lake
Serve data for application and analytics through a variety of technologies such as SQL, Server Synapse, CosmosDB and TSI
Build transformation pipelines into dimensions and facts and therefore a strong knowledge of standard BI concepts is mandatory
Build stream pipelines leverage IoT Hub, Event Hub, Databricks streaming and other Azure stream technologies.
Work in a fluid environment with changing requirements whilst maintaining absolute attention to detail.

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Senior Network Engineer - L3 or L4 - Contract - Onsite
Location: Johannesburg
Salary: R650 - 730 per hour
senior Network engineer - L3 or L4 - Contract - Onsite
1 day ago

Intermediate SQL Developer (Initial 1 year contract)
Location: Johannesburg
Salary: Annually
A client that has developed a software solutions package (administration and accounting), tailored specifically for the retirement fund industry, is looking for an Intermediate SQL Developer to join their team on an initial 1 year contract (possibility of going perm thereafter).
1 day ago

Intermediate React Native Developer
Location: Johannesburg
Salary: Annually
A real-time data capturing solutions firm, in the north of Johannesburg, is looking for a senior React Developer to join their team, where you will be involved in designing and re-developing existing platforms.
1 day ago

Head of Data Strategy & Operations
Location: Midrand
Salary:
Are you a visionary leader in data operations and digital transformation? A leading digital service provider in the higher education sector, is seeking a dynamic Head of data Strategy & Operations to drive its data management initiatives and oversee strategic growth through digital services. BCom degree essential. Also must have corporate experience in marketing and sales, very driven, present...
1 day ago

Finance & HR Administrator
Location: Pretoria
Salary: Monthly
Luxury Hotel in the Kameeldrift area is looking for an experienced Finance & HR Administrator to provide administrative support to both finance and HR department.
1 day ago

SAP Functional Consultant (Senior) - 3139
Location: Centurion
Salary: Monthly
A client in the automobile industry is looking for a SAP MM Functional Consultant (senior)
1 day ago

Financial Accountant
Location: Johannesburg
Salary: R500 000
Financial Accountant – JHB NorthJoin a Game-Changer in the Financial Services Industry (Short Term Insurance) as a Financial AccountantReady to take your career to the next level? We’re on the hunt for a senior Financial Accountant to join an innovative, forward-thinking leader in the Investments sector of the Financial Services industry. This is not just another accounting role, this ...
1 day ago

Enterprise Architect
Location: Johannesburg
Salary:
A leading organisation in the conservation and operations sector is seeking a highly skilled Enterprise Architect to drive the design and integration of complex systems across remote and hybrid-hosted environments. The successful candidate will play a key role in bridging the gap between technical teams, system vendors, and business system owners.
1 day ago

Business Development Portfolio Lead - Banking
Location: Johannesburg
Salary:
Business Development Portfolio Lead - BankingJob Level: Middle ManagementReports to: Executive: Strategic Relations and Education Role Objective:Business Development Portfolio Lead - will be responsible for expanding requires proactively engaging with potential clients to introduce them to the full range of with the internal value chain of the business. The ideal candidate will be a strategic...
1 day ago

Finance Business Partner
Location: Johannesburg
Salary: Negotiable
A dynamic opportunity has arisen for an experienced and driven Finance Business Partner to take the lead in financial reporting, performance analysis, and strategic evaluation within a well-established organisation in the financial services sector.
1 day ago

2022_51 Senior Data Engineer

Similar jobs you might be interested in:

Create a free job alert for 2022_51 Senior Data Engineer in Johannesburg