AWS Data Engineer and Information Team Lead

Recruiter:

PBT Group

Job Ref:

1079874713

Date posted:

Thursday, June 2, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
-

JOB DESCRIPTION:

PBT Group has a requirement for a Hadoop Developer to interpret requirements provided by business and produce effective Big Data solutions.

The Hadoop Data and Information specialist will be responsible to design, develop and support application solutions with focus primarily on Hadoop for a Financial Services Environment. This role will play a vital role in building new data pipelines from various structured and unstructured sources into Hadoop. Must be eager to wear multiple hats and be capable of picking up new technologies at a fast pace.

Programming exposure for code transformations and integrating the big data solution with existing systems. Develop information solutions from a variety of sources for both structured and unstructured data. Technical ownership of Big Data solutions for structured and unstructured data.

DUTIES:

Develop and implement big data models and solutions
Design and implement ETL methodologies and technologies and the integration with big data
Conduct root cause analysis on production issues
Technical leadership of entire information management process of both structured and unstructured data
Provide ongoing support and enhancement to ETL system
Optimization and the information solutions
Implementing machine learning algorithms
Configuration of the Hadoop infrastructure and environment for optimal performance
Integrate with statistical and actuarial analysts to build models
Producing relevant technical documentation and specifications
Estimate time and resource requirements for business requirement
Integration of big data solutions with existing reporting and analytical solutions
Develop data processing functions (DPF’s) using Java and Python

SKILLS REQUIRED:

Big Data Developer
Hadoop based architecture with Hortonworks Data Flow & Data Platform clusters
HDF, HBase, Hive, Spark, Storm
AWS experience would be a major advantage.

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Data Scientist
Location: Johannesburg
Salary:
Are you a talented data Scientist seeking an exciting opportunity to revolutionize the banking industry? Join a dynamic team at a leading financial institution as we harness the power of data to drive innovation and enhance customer experiences!
7 days ago

Group Accounting and Reporting Manager
Location: Johannesburg
Salary: 1200000 Annually
Are you an experienced CA(SA) eager for an exciting opportunity in the vibrant Automotive industry in North Johannesburg? Your search ends here! Our client is actively looking for a skilled Group Accounting and Reporting Manager to join their team.
20 days ago

Data Engineer
Location: Johannesburg
Salary: Monthly
A client in the software development space requires an experienced data engineer to join their team. Your main responsibility will be designing, developing, and maintaining data warehouses and data lake solutions for clients.
4 days ago

Senior Full Stack C# Developer (Blazor/Angular) – Johannesburg (Remote) – up to R850k Per Annum
Location: Johannesburg
Salary: 850000
senior Full Stack C# Developer (Blazor/Angular) – Johannesburg (Remote) – up to R850k Per Annum
18 days ago

NEW WORK IN: Senior C# Full Stack Software Engineer (Angular 10+) – Bedfordview (IN-OFFICE WITH HYBRID AFTER THREE MONTHS) – @ R1.2mil CTC P.A
Location: Pretoria
Salary: 1200000
NEW WORK IN: senior C# Full Stack Software engineer (Angular 10+) – Bedfordview (IN-OFFICE WITH HYBRID AFTER THREE MONTHS) – @ R1.2mil CTC P.A
27 days ago

Senior C# Developer (Back End) – Centurion (Hybrid) – up to R1.18mil p/a
Location: Pretoria
Salary: 1180000
senior C# Developer (Back End) – Centurion (Hybrid) – up to R1.18mil p/a
27 days ago

Auditors
Location: Johannesburg
Salary: market related paid in Euros
Auditors chartered accountants external auditors
17 days ago

Junior Factory Manager
Location: Gauteng
Salary: Salary; Medical aid; pension/provident; 13th cheque; incentives
Seeking an experienced Jnr Factory Manager with a min engineering degree in Metallurgy, Chemical, Mechanical, or Electrical. Solid experience in manufacturing/factory environment. Must have excellent maintenance skills. Demanding role, must have 3-5 years experience in similar position.
20 days ago

Deputy Factory Manager
Location: Gauteng
Salary: Basic salary; Profit share; Vehicle; Fuel; Medical Aid; Pension/provident; Disability; Cell phone
Seeking Dep Factory Manager expert with a min degree in Mechanical or Electrical engineering (or a similar field). Analytical thinker with solid managerial and manufacturing background. Operation management skills and team development skills. Demanding role. Excellent career opportunity. Vivacious and robust individuals are required.
20 days ago

JavaScript Developer
Location: Johannesburg
Salary: Monthly
My client is urgently looking for talented JavaScript Developers, with React Native experience, to join their development team.
7 days ago

Senior Big Data / Hadoop / AWS Data Engineer and Information Team Lead

Similar jobs you might be interested in:

Create a free job alert for Senior Big Data / Hadoop / AWS Data Engineer and Information Team Lead in Johannesburg