SUMMARY:
DESIGN, develop and implement scalable and robust processes to support cloud applications and solutions as the next Data Engineer sought by a leading Supply Chain Group.
POSITION INFO:
ENVIRONMENT:
DESIGN, develop and implement scalable and robust processes to support cloud applications and solutions as the next Data Engineer sought by a leading Supply Chain Group. This role includes extracting, transforming, and consolidating data, developing and maintaining a data platform to support Foods Business Capabilities and thus ensure optimal cloud solutions performance and stability through support and continuous improvements. The successful incumbent will require a Bachelor’s Degree in Computer Science/Business Informatics/Mathematics or equivalent with 4-5 years’ work experience in a similar role & possess a solid understanding of data structures, algorithms, and software design. You must also be proficient with Redshift, Athena, Glue, Lambda, Python, Scala, Java, C++, Apache Spark, AWS, Azure, GCP, Git and SVN. Please note this is an Employment Equity role ONLY.
DUTIES:
- Assist in designing and implementing scalable and robust processes for ingesting and transforming large data sets.
- Assist in the design and implementation of data pipelines from a variety of data sources and support the maintenance thereof.
- Ingest large, complex data sets that meet functional and non-functional requirements.
- Enable the business to solve the problem of working with large volumes of data in diverse formats, and in doing so, enable innovative solutions.
- Build bulk and delta data patterns for optimal extraction, transformation, and loading of data.
- Support the organisations cloud strategy and alignment to data architecture and data governance.
- Engineer data in the appropriate formats for downstream consumption for analytics or Enterprise applications.
- Assist in the development of APIs to expose the data to Enterprise Applications and 3rd party vendors.
- Assist in identifying, designing and implementing robust process improvement activities to drive efficiency and automation for greater scalability. This includes looking at new solutions and new ways of working and being on the forefront of emerging technologies.
- Work with various stakeholders across the organisation to understand data requirements and apply technical knowledge of data management to solve key business problems.
- Provide support in the operational environment with all relevant support teams for data services.
- Create and maintain functional requirements and system specifications in support of data architecture and detailed design specifications for current and future designs.
- Support test and deployment of new services and features.
REQUIREMENTS:
Qualifications –
- Bachelor’s Degree in Computer Science, Business Informatics, Mathematics, Statistics, Engineering, or equivalent.
Experience/Skills –
- 4-5 Years of relevant Data Engineering experience.
- Strong understanding of data structures, algorithms, and software design.
- Experience with structured and unstructured data, different data stores, and traditional RDBMS and data warehouses.
- Proficiency in cloud tools such as Redshift, Athena, Glue and Lambda.
- Good programming, performance tuning and troubleshooting skills using programming languages such as Python, Scala, Java and C++.
- Practical experience with Apache Spark and cloud services (e.g., AWS, Azure, GCP).
- Experience with Version Control systems like Git and SVN.
- Experience with DevOps practices, including CI/CD and Infrastructure as Code.
- Exposure to AWS Platform environment such as CodePipeline, CodeBuild, CodeDeploy and Management Console.
Specialised Skills:
Data Engineering: Experience with Big Data, ETL, and Data Management processes.
Platform Engineering: Experience with cloud platform development and maintenance.
Cloud Engineering: Experience with cloud architecture and API development.
DevOps: Experience with DevOps practices, architecture, and operation.
ML Engineering: Experience with Machine Learning frameworks and model integration.
Data Visualisation: Experience with data visualisation tools and techniques.
ATTRIBUTES:
- Excellent verbal and written communication skills.
- Strong analytical and problem-solving abilities.
- Ability to work well in an agile, collaborative team environment.
- Continuous learning mindset and commitment to professional development.
- Innovative thinking and adaptability to changing business needs.
While we would really like to respond to every application, should you not be contacted for this position within 10 working days please consider your application unsuccessful.
COMMENTS:
When applying for jobs, ensure that you have the minimum job requirements. OnlySA Citizens will be considered for this role. If you are not in the mentioned location of any of the jobs, please note your relocation plans in all applications for jobs and correspondence. Apply here
