Senior Data Engineer

SUMMARY:
-

POSITION INFO:

Job Summary:

• To lead the design, development, and governance of the company’s enterprise data infrastructure, transitioning from the current SSIS-based ETL environment to a modern, scalable, Azure-based cloud data warehouse.
• This role is strategic and collaborative, requiring close engagement with business leaders, BI developers, analysts, and IT teams to ensure data solutions are business-aligned, governance-compliant, and ready for advanced analytics.
• As a senior member of the team, the role will mentor junior data engineers, influence architectural decisions, and champion best practices across the organisation.

Key Responsibilities:

Data Warehouse Architecture & Leadership

• Lead the architectural design and implementation of the new Azure-based data warehouse.
  Oversee the optimisation of the existing SSIS-based ETL environment during the transition phase.
  Establish long-term data platform strategy in collaboration with BI and IT leadership.

Data Governance & Compliance

• Own the data governance framework, ensuring standards for data quality, security, lineage, and access control are embedded in all solutions
• Partner with compliance and legal teams to meet regulatory requirements for data storage and processing.
• Champion the use of data cataloguing and metadata management tools.

Collaboration & Stakeholder Engagement

• Work directly with senior stakeholders across business units to translate needs into actionable technical solutions.
• Serve as the primary point of contact between the BI team, engineering, and external vendors for data infrastructure matters.
• Facilitate workshops, architectural reviews, and cross-team solution design sessions.

Technical Delivery & Mentorship

• Lead the build, testing, and deployment of robust ETL/ELT pipelines for multi-source integration.
• Ensure smooth migration of historical and real-time data to the new warehouse with minimal downtime.
• Mentor and support junior and mid-level engineers in technical best practices and solution delivery.

Performance Monitoring & Continuous Improvement

• Implement platform monitoring solutions to track and optimise performance.
• Drive continuous improvement in architecture, tooling, and governance processes

Required Skills and Qualifications:

EDUCATION

Mandatory:

• Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or related field.

Preferred:

• Master’s degree in Data Engineering, Computer Science, or related discipline.
• Business Management or Project Management certification to support stakeholder engagement and project oversight.
• Azure Data Engineer Associate or equivalent cloud certification.
• Data governance certification (DAMA, DCAM, or equivalent).

EXPERIENCE & SKILLS/PHYSICAL COMPETENCIES

Technical Experience

Mandatory:

• 5+ years in data engineering, with experience in data warehouse design and development.
• Strong hands-on experience with SSIS for ETL processes.
• Proven expertise in Azure Data Platform components (Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Data Lake Storage).
• Strong SQL skills (T-SQL preferred).
• Experience implementing data governance principles, including data quality frameworks, security/access controls, and metadata management.
• Experience with data modeling (Kimball/Star Schema/Snowflake).
• Proficient in performance tuning and troubleshooting data processes.

Preferred:

• Experience with Python or other scripting languages for data processing and automation.
• Knowledge of Azure Purview or other data catalog solutions.
• Knowledge of Azure DevOps, CI/CD pipelines for data solutions.
• Power BI integration experience with Azure datasets.

Physical & Technical Competencies

• Ability to design and optimize complex ETL/ELT workflows.
• Strong understanding of data governance frameworks and compliance requirements.
• Proficient in working with large-scale data sets and high-volume data pipelines.
• Strong problem-solving skills with a focus on automation and efficiency.
• Proficient in version control tools (e.g., Git).
• Expertise in optimising data platforms for scale and performance.
• Ability to manage competing priorities across multiple projects