BI Big Data Engineer – Johannesburg – up to R950k per annum

SUMMARY:
BI Big Data Engineer – Johannesburg – up to R950k per annum

POSITION INFO:

An expert in integrated corporate benefits well known for outstanding usability and integration within the banking space is looking for a specialist capable of working within the capacity of a BI Big Data Engineer who will be responsible for building and maintaining data pipelines for new data in a variety of formats using an optimal orchestration method into a modern data platform. The candidate will have deep data engineering skills with experience using different programming languages and working comfortably on Linux or Windows environments.

Responsibilities:

• Perform data analysis and normalisation to create features for machine learning applications.
• Ingest data into the data environment in batch or real-time
• Build Meta-data driven data integration solutions
• Develop and maintain APIs for seamless integration of modern data platform with other systems.
• Productise Machine Learning models on a Linux containerized environment to ensure scalability and reliability.
• Administer Linux operating system and exhibit a strong programming experience.
• Utilise distributed programming techniques on a cluster of servers for optimised data processing.
• Collaborate with teams to advise on optimal data system architecture for data warehousing, data lakes, and lakehouses.
• Conduct root cause analysis on production issues surrounding data pipelining and execution
• Technical leadership of entire information management process of both structured and unstructured data
• Optimisation of the solutions for loading and transforming multi structure data
• Create comprehensive technical documentation for data systems, APIs, and processes.

Qualifications and Experience:

• National Diploma /BTech/Mtech IT or BSC/MSC Computer Science
• 5 years plus experience
• Python, Java, JavaScript, C# (Advanced)
• Spark SQL and R programming (Advanced)
• Building and maintaining API(s) and Webhooks (Advanced)
• Google Analytics data (Advanced)
• Real-time streaming and batch orchestration (Advanced)
• Object Oriented Programming (Advanced)
• SQL programming (Advanced)
• Data Warehouse, Data Lake and Lakehouse (Advanced)
• Git versioning of code (Advanced)

The Reference Number for this position is NG58552 which is a Permanent position in Sandton, Johannesburg offering a salary negotiable between R750k to R950k CTC based on experience and skills. E-mail Nokuthula on nokuthulag@ e-Merge.co.za or call her for a chat on to discuss this and other opportunities.

Are you ready for a change of scenery? e-Merge IT recruitment is a niche recruitment agency. We offer our candidates options so that we can successfully place the right people with the right companies, in the right roles. Check out the e-Merge IT website for more great positions.

Do you have a friend who is a developer or technology specialist? We pay cash for successful referrals!