Data Engineer

SUMMARY:
Looking for a Data Engineer in Centurion.

POSITION INFO:
The Data Engineer help stakeholders understand the data through exploration, and build and maintain secure and compliant data processing pipelines by using different tools and techniques. You maintain various data services and frameworks to store and produce cleansed and enhanced datasets for analysis.
This includes data store design using different architecture patterns based on business requirements.

Requirements:

• BCom Information Technology degree
• Data Engineering Tools (Azure Data Engineer ) certification
• ITIL or COBIT certification
• At least 2-4 years working exp in IT environment
• Exp with Phyton and SQL
• Exp with data pipelines, data lakes and Data Warehouses
• ETL processes
• Implement CI/CD pipelines

• Ensure the managed data (business intelligence) is fit for purpose relative to customer needs
• Design and develop end-to-end data solutions on the Data platform, ensuring optimal performance, scalability, and reliability.
• Collaborate with data architects, data analysts, and business stakeholders to understand data requirements, define data models, and implement data integration solutions.
• Implement data pipelines, data lakes, and data warehouses using.
• Develop and maintain ETL/ELT processes to extract, transform, and load data from various sources into Azure data platforms.
• Design and optimize data schemas, tables, and indexes for efficient data storage and retrieval.
• Perform data profiling, data cleansing, and data quality assessments to ensure data accuracy and consistency.
• Develop and implement data security and privacy measures, ensuring compliance with relevant regulations (e.g., POPIA).
• Monitor and troubleshoot data pipelines, identifying and resolving performance bottlenecks, data inconsistencies, and system issues.
• Collaborate with DevOps teams to implement CI/CD pipelines and ensure smooth deployment and release management of data solutions.
• Stay up to date with the latest data technologies, trends