Lead / Senior Azure Data Engineer

SUMMARY:
Lead / Senior Azure Data Engineer required for remote work opportunity.

POSITION INFO:

Purpose:

Perform a leadership role across various projects. Technology focussed individual able to design, configure, develop, deploy and support data and analytics application. Must be able to work in a fluid environment with changing requirements whilst maintaining absolute attention to detail. The data and analytics solution will be developed using wide range of azure technology services serving to varied architecture pattern needs.

Key Responsibilities:

• Develop, test, operate and support the end-to-end batch, near real time data and analytics solution.
• Build stream pipelines and leverage IoT Hub, Event Hub, Databricks streaming and other Azure stream technologies.
• Be able to develop quality Python/PySpark and adopt/implement software development best practices to ensure high quality standards are met.
• Performs thorough code reviews of fellow developers and ensure quality code.
• Use DevOps knowledge, cloud and data engineering expertise to operate, monitor and troubleshoot the pipelines / code bases.
• Effectively test and adopt testing frameworks to embed and codify tests.
• Take technical ownership for the implementation of the solution at an allocated mining site.
• Clearly communicate and explain concepts to senior stakeholders and key teams during sprint reviews and other programme meetings.
• Have a clear understanding of agile best practices and proactively collaborate and support the team if needed during the whole sprint cycle.
• Document technical implementation details and artefacts created as part of the process.
• Serve data for application and analytics through a variety of technologies such as Delta Lake, SQL DB, CosmosDB and TSI.
• Build transformation pipelines into dimensions and facts and therefore a strong knowledge of standard BI concepts is mandatory.

NB! This job is now closed. You can apply for other jobs by uploading your CV.