Senior Lead Data Engineer

 

Recruiter:

PBT Group

Job Ref:

PBT3744245156

Date posted:

Tuesday, October 4, 2022

Location:

Johannesburg, South Africa

Salary:

market related


JOB SUMMARY:
To complement the existing cross-functional team, PBT Group is looking for a Senior Lead Data Engineer who will design and implement scalable and robust processes to support the data engineering capability.

JOB DESCRIPTION:

To complement the existing cross-functional team, PBT Group is looking for a Senior Lead Data Engineer who will design and implement scalable and robust processes to support the data engineering capability. This role will be responsible for implementing and supporting large-scale data ecosystems across the Group. This thought leader will use best practices in cloud engineering, data management and data storage to continue our drive to optimize the way that data is stored, consumed and ultimately democratized. The incumbent will also collaborate with stakeholders across the organisation with use of the Data Engineering practices to facilitate the improvement in the way that data is stored and consumed.

  • Design and implement scalable and robust processes for ingesting and transforming complex datasets.
  • Designs, develops, constructs, maintains and supports data pipelines for ETL from a multitude of sources.
  • Creates blueprints for data management systems to centralize, protect, and maintain data sources.
  • Focused on data stewardship and curation, the data engineer enables the data scientist to run their models and analyses to achieve the desired business outcomes
  • Ingest large, complex data sets that meet functional and non-functional requirements.
  • Enable the business to solve the problem of working with large volumes of data in diverse formats, and in doing so, enable innovative solutions.
  • Design and build bulk and delta data lift patterns for optimal extraction, transformation, and loading of data.
  • Ensure that all data solutions support the organisation’s cloud strategy and aligns to the data architecture and governance including implementation of these data governance practices.
  • Engineer data in the appropriate formats for downstream customers, risk and product analytics or enterprise applications.
  • Development of API’s for returning data to Enterprise Applications.
  • Identify, design and implement robust process improvement activities to drive efficiency and automation for greater scalability. This includes looking at new solutions and new ways of working and being on the forefront of emerging technologies
  • Work with various stakeholders across the organisation to understand data requirements and apply deep technical knowledge of data management to solve key business problems.
  • Provide specialised support in the operational environment with all relevant support teams for data services.
  • Effective management of demand across the various data streams and use cases.
  • Create and maintain functional requirements, system process and system specifications in support of data architecture and detailed design specifications for current and future designs.
  • Support test and deployment of new services and features.
  • Provides technical leadership to a team of intermediate and junior data engineers, and Quality Assurance engineers.
  • Ensure robust collaboration and alignment with Group Solution, Data and Security Architects and manages engagement with business stakeholders effectively.

ROLE & QUALIFICATIONS REQUIREMENTS

  • Matric, with a degree in Computer Science, Business Informatics, Mathematics, Statistics, Physics or Engineering.
  • 5+ years of data engineering experience
  • 5+ years of experience with any data warehouse technical architectures, ETL/ELT, and reporting/analytics tools including , but not limited to , any of the following combinations: (1) SSIS and SSRS, (2) ETL Frameworks, (3) Data conformance, (4) Caching, (5) Hadoop (6), AWS data builds
  • The candidate having DBA ability and knowledge across at least 2 platforms (example: TSQL, SAS, PSQL, IBM VSAM, DynamoDB and DB2) will also be beneficial.
  • Tends towards Full-Stack engineer with experience across a broad spectrum of technology domains, technologies and Engineering practices
  • Highly Proficient in at least one of Python, Java, C/C++ or similar
  • Should be Proficient in bash/ksh, perl, SQL
  • Experience with R, Python, AWS, XML, json, cron
  • Experience with designing and implementing Cloud (AWS) solutions including full use of all APIs available.
  • Experience with Dev/OPS architecture, implementation and operation.
  • Knowledge of Engineering and Operational Excellence using standard methodologies. Best practices in software engineering, data management, data storage, data computing and distributed systems to solve business problems with data.
  • Experience in applying SAFe/Scrum/Kanban methodologies.
  • Knowledge and understanding of business process management lifecycle which covers the design, modelling, execution, monitoring, and optimization as well as business process reengineering.
  • Strong problem solving skills demonstrating the ability to exercise judgment in solving technical, operational, and organizational challenges.
  • Identify issues proactively and present solutions and options leading to a stable and resilient resolution.
  • Expert programming, performance tuning and troubleshooting skills, using the latest popular programming languages such as python, scala, java and suite of Microsoft languages C# and F# preferable.

COMPETENCIES:

  • Collaboration and precise communication
  • Establishing and maintaining strong working relationships with stakeholders across all levels
  • Strategic thinking
  • Leading with Influence
  • Innovating and being curious about technology
  • Quality in execution and outcomes
  • Customer First attitude
  • Personal Mastery (Learning)
  • Problem Solving
  • Data Management
  • Working with ambiguity
  • Working independently and within a team

 

NB! This job is now closed. You can apply for other jobs by uploading your CV.



 

 

 

Similar jobs you might be interested in:

Senior Data Engineer / Database Developer
Location: Centurion
Salary: Monthly
A consultancy firm is looking for an intellectual, solutions orientated data engineer / database Developer, to be based in Gauteng. A passion for data analysis, -manipulation and -modelling is an absolute must. Exceptional communication and the ability to be flexible will assist the individual to be successful in this type of industry.
Today


AWS Data Engineer - Johannesburg – up to R700 per hour
Location: Pretoria
Salary: 1344000
AWS data engineer - Johannesburg – up to R700 per hour
Today


Senior data engineer
Location: Johannesburg
Salary:
A well-known marketing organization with branches in multiple overseas countries is looking for a Senior data engineer to join their organization at their Johannesburg branch. The Senior data engineer will be required to work on a hybrid module. Working within the data Science Team, the data engineer will be responsible for the organization's data structure.
Today


Data Engineer
Location: Johannesburg
Salary:
A well-known marketing organization with branches in multiple overseas countries is looking for a Senior data engineer to join their organization at their Johannesburg branch. The Senior data engineer will be required to work on a hybrid module. Working within the data Science Team, the data engineer will be responsible for the organization's data structure.
Today


Data Engineer
Location: Johannesburg
Salary:
Responsible for maintaining the data warehouse through design and implementation of ETL/ELT methodologies and technologies, as well as providing maintenance and support to our ETL and ML environments. To ensure optimal performance, he conducted root cause analysis on production issues and provided technical leadership throughout the entire information management process of both structured and unst...
3 days ago


Azure Data Engineer
Location: Pretoria
Salary:
2679_Azure data engineer (Advanced)Location: GautengContract Term: 1 May 2024 - 31December 2026Applications Submissions: ChantelleT@jordanhr.co.za
6 days ago


Entry Level Data Engineer
Location: Johannesburg
Salary:
An international data Analytics company based in Johannesburg is actively recruiting for a Junior data engineer. The company is small and intimate with a massive impact in the market. Your main focus will be to implement data architecture within the company and keep up to date with data engineering.
8 days ago


AWS Data Engineer
Location: Johannesburg
Salary:
8 days ago


AWS Data Engineer
Location: Johannesburg
Salary: Hourly
An urgent need exists for an AWS data engineer position at a bank in Johannesburg. The AWS data engineer will join the Global Markets division of the bank. In this role, you will work closely with the traders, quantitative analysts, and data scientists to design, develop, and maintain data infrastructure and pipelines for the global markets business. The role offers an exciting opportun...
9 days ago


Junior Data Engineer
Location: Johannesburg
Salary: 180 000 Annually
Are you ready to be part of a cutting-edge data and analytics firm that is revolutionizing the way organizations operate? This company is seeking talented individuals to join their dynamic output and delivery-focused team. As a data and analytics firm, they combine disruptive innovation methodologies with artificial intelligence to help transform organizations.
1 day ago


Create a free job alert for data engineer in Johannesburg

Enter your email address below and we will email you similar jobs when they become available:

You can cancel at any time. We will not spam you.
By giving us your email address your agree to our Terms and Conditions