Data Engineer 111

SUMMARY:
The Role:

Our client, a leader in the retail sector are in search of a  Data Engineer 111 for a 12-month fixed term contract. They require minimum 5 years in Data/Software Engineering experience.

The successful candidate must build and support data pipelines and datamarts built off those pipelines. Both must be scalable, repeatable and ...

POSITION INFO:
The Role:

Our client, a leader in the retail sector are in search of a  Data Engineer 111 for a 12-month fixed term contract. They require minimum 5 years in Data/Software Engineering experience.

The successful candidate must build and support data pipelines and datamarts built off those pipelines. Both must be scalable, repeatable and secure. The Data Engineer helps to facilitate gathering data from a variety of different sources, in the correct format, assuring that it conforms to data quality standards and assuring that downstream users can get to that data timeously. This role functions as a core member of an agile team.

Data Engineering is a technical job that requires substantial expertise in a broad range of software development and programming fields. These professionals have a knowledge of data analysis, end user requirements and business requirements analysis to develop a clear understanding of the business need and to incorporate these needs into a technical solution. They have a solid understanding of physical database design and the systems development lifecycle. This role must work well in a team environment.

• Capability to architect highly scalable distributed systems, using different open source tools
• Big Data batch and streaming tools
• Talend (1 year)
• AWS: EMR, EC2, S3 (1 year)
• Python (1 year) * PySpark or Spark (1 year) advantageous
• Business Intelligence Data modelling (3 years)
• SQL (3 years)

• Bachelorâ??s degree in computer science, computer engineering, or equivalent work experience
• AWS Certification at least to associate level

• 5+ years Data engineering or software engineering
• 3-5 years demonstrated experience leading teams of engineers
• 2+ years Big Data experience
• 5+ yearsâ?? experience with Extract Transform and Load (ETL) processes
• 2+ years Could AWS experience
• At least 2 years demonstrated experience with agile or other rapid application development methods - Agile exposure, Kanban or Scrum
• 5 years demonstrated experience with object-oriented design, coding and testing patterns as well as experience in engineering (commercial or open source) software platforms and large-scale data infrastructures
• Creating data feeds from on-premise to AWS Cloud (2 years)
• Support data feeds in production on break fix basis (2 years)
• Creating data marts using Talend or similar ETL development tool (4 years) Manipulating data using python and pyspark (2 years)
• Processing data using the Hadoop paradigm particularly using EMR, AWSâ??s distribution of Hadoop (2 years)
• Extensive knowledge in different programming or scripting languages
• Expert knowledge of data modeling and understanding of different data structures and their benefits and limitations under particular use cases

NB! This job is now closed. You can apply for other jobs by uploading your CV.