Senior Data Engineer with AWS G1775

SUMMARY:
We are currently recruiting for a Senior Data Engineer with extensive experience in AWS.  The ideal candidate will have a minimum of 8 years experience in an expert level role.

• Expertise in ETL optimization, designing, coding, and tuning big data processes using Apache Spark. 
• Experience with building data pipelines and applications to stream and process datasets at low latencies. 
• Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data. 
• Sound knowledge of distributed systems and data architecture (lambda)- design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning of high-level data structures. 
• Experience designing and supporting large-scale distributed systems in a production environment 

• Lambda 
• DynamoDB 
• Param Store 
• Secrets Manager 
• Athena 
• Glue 
• CloudWatch 
• Step Function 
• SNS 
• Code Build/Pipeline 
• CloudFormation 
• S3 

• Python 3x 
• SQL 
• Py Spark 
• Boto3 
• Terraform 
• ETL 
• Docker 
• Linux / Unix 
• Big Data 
• Oracle/PostgreSQL 
• Powershell / Bash 

POSITION INFO:
Role Tasks: 

• Coordination between development and support environments 
• Assisting with the business case 
• Planning and monitoring 
• Eliciting requirements 
• Requirements organisation 
• Translating and simplifying requirements 
• Requirements management and communication 
• Requirements analysis 
• Document requirements in appropriate format depending on methodology followed 
• Assist with identification and management of risks 

NB! This job is now closed. You can apply for other jobs by uploading your CV.