AWS Data Engineer

PBT Group has an opportunity for an Intermediate Data Engineer. The Data Engineer must have experience in migrating legacy to AWS Lambda.

Duties:

• Assist in designing and implementing scalable and robust processes for ingesting and transforming datasets.
• Design, implement and support the creation and maintenance of data pipelines from a multitude of sources.
• Ingest large, complex data sets that meet functional and non-functional requirements.
• Enable the business to solve the problem of working with large volumes of data in diverse formats, and in doing so, enable innovative solutions.
• Design and build bulk and delta data lift patterns for optimal extraction, transformation, and loading of data.
• Supports the organisation’s cloud strategy and aligns to the data architecture and governance including the implementation of these data governance practices.
• Engineer data in the appropriate formats for downstream customers, risk and product analytics or enterprise applications.
• Development of API’s for returning data to Enterprise Applications.
• Assist in identifying, designing and implementing robust process improvement activities to drive efficiency and automation for greater scalability. This includes looking at new solutions and new ways of working and being on the forefront of emerging technologies.
• Work with various stakeholders across the organisation to understand data requirements and apply technical knowledge of data management to solve key business problems.

• Provide support in the operational environment with all relevant support teams for data services.
• Create and maintain functional requirements and system specifications in support of data architecture and detailed design specifications for current and future designs.
• Support, test and deployment of new services and features.
• Design and Implement actionable Alerts, Monitoring and Observability using various toolsets that support our resilience including DR activities
• Design and build Data validation and reconciliation alerts from source

Required Skills:

• Strong modular system design skills.
• Conventional and data warehouse modeling skills are required, in order to understand the various data models and to define the mappings between them.
• System analysis and design skills are necessary to design and document the data extractions and transformations.
• Expert knowledge of the capabilities of the ETL tools being used, to know what their capabilities and shortcomings are – in order to exploit or avoid those aspects in the ETL program designs.
• Good organisation, planning and basic management skills.
• Good interpersonal and communication skills.
• Decision making and problem solving skills.

Required Qualifications / Training:

• Relevant data warehouse and BI solution training is essential.
• B.Sc. or related degree is advantageous.