Senior Data Engineer with AWS G1775

Recruiter:

Open Source (Pty) Ltd

Job Ref:

CPT001124/PV

Date posted:

Tuesday, December 21, 2021

Location:

Midrand, South Africa

SUMMARY:
We are currently recruiting for a Senior Data Engineer with extensive experience in AWS. The ideal candidate will have a minimum of 8 years experience in an expert level role.

Expertise in ETL optimization, designing, coding, and tuning big data processes using Apache Spark.
Experience with building data pipelines and applications to stream and process datasets at low latencies.
Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.
Sound knowledge of distributed systems and data architecture (lambda)- design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning of high-level data structures.
Experience designing and supporting large-scale distributed systems in a production environment

AWS Solid understanding of Components - VPC | IAM. Above average experience/understanding of AWS Components:

Lambda
DynamoDB
Param Store
Secrets Manager
Athena
Glue
CloudWatch
Step Function
SNS
Code Build/Pipeline
CloudFormation
S3

Strong experience/understanding of:

Python 3x
SQL
Py Spark
Boto3
Terraform
ETL
Docker
Linux / Unix
Big Data
Oracle/PostgreSQL
Powershell / Bash

POSITION INFO:
Role Tasks:

Coordination between development and support environments
Assisting with the business case
Planning and monitoring
Eliciting requirements
Requirements organisation
Translating and simplifying requirements
Requirements management and communication
Requirements analysis
Document requirements in appropriate format depending on methodology followed
Assist with identification and management of risks

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Senior Data Engineer
Location: South Africa
Salary: Annually
Start 2026 like the Cape Town Cycle Tour—fast, focused, and leading the pack! We need a senior data engineer who can architect pipelines that power AI-driven insights. This is your chance to ride the wave of innovation where AI writes 40% of the code, and you steer the strategy.
11 days ago

Senior Data Engineer
Location: Johannesburg
Salary:
44 days ago

Senior Data Engineer
Location: Centurion
Salary: Market related
We are seeking a senior data engineer to join our team based in Centurion. Onsite, contract.
49 days ago

Data Engineer (Senior) 3245
Location: Menlyn
Salary:
data engineer (senior)long-term contract
4 days ago

Senior Specialist:Data Analytics
Location: Johannesburg
Salary: 1200000 Annually
senior Specialist: data Analytics Opportunity
5 days ago

Senior Data Analyst
Location: Johannesburg
Salary: Hourly
Role Summary:The senior data Analyst will play a pivotal role in transforming data into actionable insights that drive strategic decision-making across the organisation. This role requires a strong analytical mindset and proficiency in data visualisation. The ideal candidate will bridge the gap between data and business. Responsibilities:Develop and maintain dashboards and reports to support ...
28 days ago

Senior Data Scientist
Location: Johannesburg
Salary: Market Related Monthly
Role Summary:The senior data Analyst will play a pivotal role in transforming data into actionable insights that drive strategic decision-making across the organisation. This role requires a strong analytical mindset and proficiency in data visualisation. The ideal candidate will bridge the gap between data and business. Responsibilities:Develop and maintain dashboards and reports to support ...
28 days ago

Senior Data Scientist
Location: Johannesburg
Salary:
43 days ago

Senior Data Warehouse Developer
Location: Centurion
Salary: Monthly
Psybergate is an IT company that builds bespoke software solutions and provides highly skilled resources to itsÂ clients.We are looking for a senior data Warehouse Developer to join our financial services client based in Centurion.
3 days ago

Senior Salesforce Engineer / Developer
Location: Johannesburg
Salary:
Today

Senior Data Engineer with AWS G1775

Similar jobs you might be interested in:

Create a free job alert for Senior Data Engineer with AWS G1775 in Midrand