Senior Data Engineer

Recruiter:

Network Recruitment

Job Ref:

ITE006391/Mon

Date posted:

Thursday, October 17, 2024

Location:

Johannesburg, South Africa

Salary:

Annually

SUMMARY:
Seeking a Senior Data Engineer who will be responsible for implementation and improvements.

POSITION INFO:
Responsibilities:

Identify and Implement Process Improvements: Automate manual tasks, optimize data delivery, and enhance technology scalability.
Enhance Data Systems: Implement new features and improvements across data platforms.
Streamline CI/CD Processes: Optimize data pipelines within continuous integration and delivery workflows.
Manage Complex Data Sets: Assemble large, complex datasets to meet business and technical requirements.
Optimize ETL Processes: Ensure efficient data extraction, transformation, and loading.
Conduct Testing for Data Consistency: Develop and execute unit tests to maintain data accuracy across pipelines.
Automate Monitoring: Build and maintain automated solutions for tracking data pipeline performance and health.
Support Analytics Infrastructure: Maintain and improve reporting and analytics systems.
Ensure Data Quality and Governance: Uphold data governance standards and maintain infrastructure (AWS, database security, etc.).
Manage Metadata and Documentation: Maintain the data warehouse, data lake metadata, catalogs, and user documentation.
Ensure Best Practices in Database Management: Maintain optimal configurations, including collations, engines, and indexing strategies.

Education:

5+ years' experience as a Data Engineer
BSc Degree in Computer Science / Information system / Engineering or related or equivalent experience
At least 2 years' experience building and optimising 'big data' data pipelines, architecture and maintaining data sets
Experience in Python, SQL, AWS, PySpark (desirable), AWS, Airflow, Docker
Proficiency in manipulating, processing and architecting large disconnected data sets for analytical requirements

Apply now!

For more IT jobs, please visit />
If you have not had any response in two weeks, please consider the vacancy application unsuccessful. Your profile will be kept on our database for any other suitable roles / positions.

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Senior Data Engineer
Location: South Africa
Salary: Annually
Start 2026 like the Cape Town Cycle Tour—fast, focused, and leading the pack! We need a senior data engineer who can architect pipelines that power AI-driven insights. This is your chance to ride the wave of innovation where AI writes 40% of the code, and you steer the strategy.
18 days ago

Senior Data Engineer
Location: Johannesburg
Salary:
51 days ago

Senior Data Engineer
Location: Centurion
Salary: Market related
We are seeking a senior data engineer to join our team based in Centurion. Onsite, contract.
56 days ago

Data Engineer (Senior) 3245
Location: Menlyn
Salary:
data engineer (senior)long-term contract
11 days ago

Senior Data Scientist
Location: Johannesburg
Salary:
50 days ago

Senior Data Warehouse Developer
Location: Centurion
Salary: Monthly
Psybergate is an IT company that builds bespoke software solutions and provides highly skilled resources to itsÂ clients.We are looking for a senior data Warehouse Developer to join our financial services client based in Centurion.
10 days ago

Senior Data Science Consultant
Location: South Africa
Salary: market related Monthly
Our client is seeking a full-time senior data Science Consultant with strong data science and consulting expertise to join their dynamic and growing data Science team.Â Role and Responsibilities:As part of the company data Science team, you will report directly to the Head: data Science.As a senior data Science Consultant, you will be responsible for designing and leading data science projects; an...
7 days ago

Data Engineer
Location: Johannesburg
Salary: Annually
The data engineer will be responsible for developing data architecture and integration pipelines to ensure information accuracy, quality and accessibility.
1 day ago

Applications Developer
Location: Johannesburg
Salary: Annually
The Applications Developer will be responsible for designing, developing and supporting business-critical applications and systems integrations that strengthen digital capability. The role focuses on delivering secure, scalable, and maintainable solutions that improve business processes, enable reliable data flow and support operational excellence.
1 day ago

Database Administrator
Location: Centurion
Salary: Annually
The database Administrator will be responsible for the day-to-day maintenance, and administration of the organisation's IT infrastructure, including database management, server maintenance, cloud environment and security systems.
3 days ago