Senior Data Engineer with AWS G1775

Recruiter:

Open Source (Pty) Ltd

Job Ref:

CPT001124/PV

Date posted:

Tuesday, December 21, 2021

Location:

Midrand, South Africa

SUMMARY:
We are currently recruiting for a Senior Data Engineer with extensive experience in AWS. The ideal candidate will have a minimum of 8 years experience in an expert level role.

Expertise in ETL optimization, designing, coding, and tuning big data processes using Apache Spark.
Experience with building data pipelines and applications to stream and process datasets at low latencies.
Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.
Sound knowledge of distributed systems and data architecture (lambda)- design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning of high-level data structures.
Experience designing and supporting large-scale distributed systems in a production environment

AWS Solid understanding of Components - VPC | IAM. Above average experience/understanding of AWS Components:

Lambda
DynamoDB
Param Store
Secrets Manager
Athena
Glue
CloudWatch
Step Function
SNS
Code Build/Pipeline
CloudFormation
S3

Strong experience/understanding of:

Python 3x
SQL
Py Spark
Boto3
Terraform
ETL
Docker
Linux / Unix
Big Data
Oracle/PostgreSQL
Powershell / Bash

POSITION INFO:
Role Tasks:

Coordination between development and support environments
Assisting with the business case
Planning and monitoring
Eliciting requirements
Requirements organisation
Translating and simplifying requirements
Requirements management and communication
Requirements analysis
Document requirements in appropriate format depending on methodology followed
Assist with identification and management of risks

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Senior Data Engineer
Location: Johannesburg
Salary: Hourly
Our client, a well-established global financial services group with a long-standing presence in South Africa, operating within a highly regulated, data-driven environment, is seeking a senior data engineer to join their team on a 12-month contract. The role would ideally suit a data engineer whose contract is not being renewed and understands the contracting model. This role offers the opport...
13 days ago

Senior Data Engineer
Location: Pretoria
Salary:
We are seeking an experienced and highly skilled senior data engineer to join our growing data team. This role is responsible for designing, developing, and maintaining scalable data solutions that support business intelligence, analytics, reporting, and operational systems.As a senior data engineer, you will play a key role in building robust data pipelines, integrating data from multiple sources...
18 days ago

Senior Data Engineer
Location: Johannesburg
Salary:
54 days ago

Mid-to-Senior Network and Data Centre Engineer
Location: Centurion
Salary:
Our client within the IT space is seeking a Mid to senior Network and data Centre engineer to join their team!Salary R60 000 CTC per month Area: Centurion Purpose of the Role:The purpose of the role is to ensure the stability, security, performance and ongoing development of the organisation’s IT infrastructure, including Microsoft 365, servers, data centre, networks, backups and s...
2 days ago

Senior Data Product Engineer
Location: Johannesburg
Salary:
52 days ago

Intermediate to Senior BI Developer and Data Analyst
Location: Johannesburg
Salary: 60 000 Monthly
Role: Intermediate to senior BI Developer & data AnalystLocation: Gauteng - Sandton Our client, a successful provider of integrated, end-to-end solutions and specialist business services, is seeking a dynamic Intermediate or senior BI Developer & data Analyst. The ideal candidate is a proactive, energetic leader and a collaborative team player. This role offers a&n...
4 days ago

Senior Researcher (Smart Data Integration - 12 Month Fixed Term Contract - Wits Health Consortium)
Location: Sandringham
Salary: Monthly
Background:Wits Health Consortium (Pty) Limited ("WHC") is a wholly-owned Company of the University of the Witwatersrand, Johannesburg (South Africa) under its Faculty of Health SciencesWHC provides Faculty with a legal framework within which to operate the research and other activities necessary to support its academic objectivesIn addition, WHC offers a range of products and services to Academic...
6 days ago

Senior Data Specialist
Location: Johannesburg
Salary: 800 000 Annually
My client is looking for a data engineer to join the team — and we’re rounding up someone who knows their way around some serious data!
24 days ago

Senior Data Analyst - Finance, BI & Digital Transformation
Location: Sandton
Salary: R90k - 100k per month
senior data Analyst to lead the analysis and solution-design phases of digital transformation
62 days ago

Mid/Senior Data Support Specialist
Location: Sandton
Salary:
82 days ago

Senior Data Engineer with AWS G1775

Similar jobs you might be interested in:

Create a free job alert for Senior Data Engineer with AWS G1775 in Midrand