AWS Data Engineer and Information Team Lead

Recruiter:

PBT Group

Job Ref:

1079874713

Date posted:

Thursday, June 2, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
-

JOB DESCRIPTION:

PBT Group has a requirement for a Hadoop Developer to interpret requirements provided by business and produce effective Big Data solutions.

The Hadoop Data and Information specialist will be responsible to design, develop and support application solutions with focus primarily on Hadoop for a Financial Services Environment. This role will play a vital role in building new data pipelines from various structured and unstructured sources into Hadoop. Must be eager to wear multiple hats and be capable of picking up new technologies at a fast pace.

Programming exposure for code transformations and integrating the big data solution with existing systems. Develop information solutions from a variety of sources for both structured and unstructured data. Technical ownership of Big Data solutions for structured and unstructured data.

DUTIES:

Develop and implement big data models and solutions
Design and implement ETL methodologies and technologies and the integration with big data
Conduct root cause analysis on production issues
Technical leadership of entire information management process of both structured and unstructured data
Provide ongoing support and enhancement to ETL system
Optimization and the information solutions
Implementing machine learning algorithms
Configuration of the Hadoop infrastructure and environment for optimal performance
Integrate with statistical and actuarial analysts to build models
Producing relevant technical documentation and specifications
Estimate time and resource requirements for business requirement
Integration of big data solutions with existing reporting and analytical solutions
Develop data processing functions (DPF’s) using Java and Python

SKILLS REQUIRED:

Big Data Developer
Hadoop based architecture with Hortonworks Data Flow & Data Platform clusters
HDF, HBase, Hive, Spark, Storm
AWS experience would be a major advantage.

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Cloud Engineer
Location: Johannesburg
Salary:
We are looking for an experienced Cloud engineer to join a pioneering company that designs and develops telematics solutions for the mining industry, based in Lonehill. This is a fully in-office role, offering the opportunity to work on innovative projects that transform mining operations through advanced cloud technologies.
4 days ago

QA Engineer
Location: Johannesburg
Salary: Monthly
We are looking for a skilled QA engineer to join an in-house software department supporting a dynamic group of companies, based in Germiston. This is an exciting opportunity to gain exposure to a wide range of software development projects and contribute to delivering high-quality solutions.
4 days ago

Junior Developer
Location: Johannesburg
Salary: Monthly
We are looking for a versatile Junior Developer with hands-on experience in multiple programming languages, including Python, Java, and C#. Join an in-house software department supporting a dynamic group of companies, based in Germiston, and gain exposure to a wide range of software development projects.
4 days ago

Actuarial Analyst
Location: Johannesburg
Salary:
Are you prepared to redefine what’s possible in actuarial science? This is your chance to join a forward-thinking team where your expertise in product development, market analytics, and data insights won’t just be used – it will be celebrated. If you’ve ever wanted to combine technical modelling with innovation and help shape life insurance products that truly stand out, th...
5 days ago

Senior Data Engineer
Location: Johannesburg
Salary: Annual Salary
Are you the BEST senior data engineer??
5 days ago

Senior Network & IT Support Technician - Johannesburg
Location: Johannesburg
Salary: Market-Related
You''ll take ownership of the technical environment in Gauteng, supporting the setup, installation, and maintenance of IoT and network systems for regional clients.
5 days ago

Financial Accountant
Location: Johannesburg
Salary: 650 000 Annually
Are you a skilled Accountant with a passion for numbers and ready to hit the ground running? Our client is looking for a temporary Accountant to join a leading company in the industrial/mining industry, based in Randburg. This will be a 6-month contract and it's highly advantageous if you are immediately available.
6 days ago

Senior Internal Auditor (Operational Audits) - Remote
Location: Johannesburg
Salary: 450000 Annually
Ready to take the lead on impactful audits? Apply now and bring your insight to the forefront of operational excellence!
7 days ago

Actuarial Analyst
Location: Johannesburg
Salary:
Are you prepared to redefine what’s possible in actuarial science? This is your chance to join a forward-thinking team where your expertise in product development, market analytics, and data insights won’t just be used – it will be celebrated. If you’ve ever wanted to combine technical modelling with innovation and help shape life insurance products that truly stand out, th...
7 days ago

IT Audit Manager
Location: Johannesburg
Salary: 1100000 Annually
A leading insurance provider known for its commitment to innovation and excellence, is currently seeking a highly skilled and experienced Audit Manager specialising in data Analytics to join their dynamic team.
9 days ago

Senior Big Data / Hadoop / AWS Data Engineer and Information Team Lead

Similar jobs you might be interested in:

Create a free job alert for Senior Big Data / Hadoop / AWS Data Engineer and Information Team Lead in Johannesburg