SUMMARY:
-
POSITION INFO:
Job Summary:
- To lead the design, development, and governance of the company’s enterprise data infrastructure, transitioning from the current SSIS-based ETL environment to a modern, scalable, Azure-based cloud data warehouse.
- This role is strategic and collaborative, requiring close engagement with business leaders, BI developers, analysts, and IT teams to ensure data solutions are business-aligned, governance-compliant, and ready for advanced analytics.
- As a senior member of the team, the role will mentor junior data engineers, influence architectural decisions, and champion best practices across the organisation.
Key Responsibilities:
Data Warehouse Architecture & Leadership
- Lead the architectural design and implementation of the new Azure-based data warehouse.
Oversee the optimisation of the existing SSIS-based ETL environment during the transition phase.
Establish long-term data platform strategy in collaboration with BI and IT leadership.
Data Governance & Compliance
- Own the data governance framework, ensuring standards for data quality, security, lineage, and access control are embedded in all solutions
- Partner with compliance and legal teams to meet regulatory requirements for data storage and processing.
- Champion the use of data cataloguing and metadata management tools.
Collaboration & Stakeholder Engagement
- Work directly with senior stakeholders across business units to translate needs into actionable technical solutions.
- Serve as the primary point of contact between the BI team, engineering, and external vendors for data infrastructure matters.
- Facilitate workshops, architectural reviews, and cross-team solution design sessions.
Technical Delivery & Mentorship
- Lead the build, testing, and deployment of robust ETL/ELT pipelines for multi-source integration.
- Ensure smooth migration of historical and real-time data to the new warehouse with minimal downtime.
- Mentor and support junior and mid-level engineers in technical best practices and solution delivery.
Performance Monitoring & Continuous Improvement
- Implement platform monitoring solutions to track and optimise performance.
- Drive continuous improvement in architecture, tooling, and governance processes
Required Skills and Qualifications:
EDUCATION
Mandatory:
- Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or related field.
Preferred:
- Master’s degree in Data Engineering, Computer Science, or related discipline.
- Business Management or Project Management certification to support stakeholder engagement and project oversight.
- Azure Data Engineer Associate or equivalent cloud certification.
- Data governance certification (DAMA, DCAM, or equivalent).
EXPERIENCE & SKILLS/PHYSICAL COMPETENCIES
Technical Experience
Mandatory:
- 5+ years in data engineering, with experience in data warehouse design and development.
- Strong hands-on experience with SSIS for ETL processes.
- Proven expertise in Azure Data Platform components (Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Data Lake Storage).
- Strong SQL skills (T-SQL preferred).
- Experience implementing data governance principles, including data quality frameworks, security/access controls, and metadata management.
- Experience with data modeling (Kimball/Star Schema/Snowflake).
- Proficient in performance tuning and troubleshooting data processes.
Preferred:
- Experience with Python or other scripting languages for data processing and automation.
- Knowledge of Azure Purview or other data catalog solutions.
- Knowledge of Azure DevOps, CI/CD pipelines for data solutions.
- Power BI integration experience with Azure datasets.
Physical & Technical Competencies
- Ability to design and optimize complex ETL/ELT workflows.
- Strong understanding of data governance frameworks and compliance requirements.
- Proficient in working with large-scale data sets and high-volume data pipelines.
- Strong problem-solving skills with a focus on automation and efficiency.
- Proficient in version control tools (e.g., Git).
- Expertise in optimising data platforms for scale and performance.
- Ability to manage competing priorities across multiple projects
