AWS Data Engineer and Information Team Lead

Recruiter:

PBT Group

Job Ref:

1079874713

Date posted:

Thursday, June 2, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
-

JOB DESCRIPTION:

PBT Group has a requirement for a Hadoop Developer to interpret requirements provided by business and produce effective Big Data solutions.

The Hadoop Data and Information specialist will be responsible to design, develop and support application solutions with focus primarily on Hadoop for a Financial Services Environment. This role will play a vital role in building new data pipelines from various structured and unstructured sources into Hadoop. Must be eager to wear multiple hats and be capable of picking up new technologies at a fast pace.

Programming exposure for code transformations and integrating the big data solution with existing systems. Develop information solutions from a variety of sources for both structured and unstructured data. Technical ownership of Big Data solutions for structured and unstructured data.

DUTIES:

Develop and implement big data models and solutions
Design and implement ETL methodologies and technologies and the integration with big data
Conduct root cause analysis on production issues
Technical leadership of entire information management process of both structured and unstructured data
Provide ongoing support and enhancement to ETL system
Optimization and the information solutions
Implementing machine learning algorithms
Configuration of the Hadoop infrastructure and environment for optimal performance
Integrate with statistical and actuarial analysts to build models
Producing relevant technical documentation and specifications
Estimate time and resource requirements for business requirement
Integration of big data solutions with existing reporting and analytical solutions
Develop data processing functions (DPF’s) using Java and Python

SKILLS REQUIRED:

Big Data Developer
Hadoop based architecture with Hortonworks Data Flow & Data Platform clusters
HDF, HBase, Hive, Spark, Storm
AWS experience would be a major advantage.

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Senior Salesforce Developer
Location: Johannesburg
Salary:
Our client is seeking a senior Salesforce Developer to join their team, based fully in-office in Bedfordview. This role is suited to a highly experienced Salesforce professional who is passionate about designing scalable architecture, leading technical solutions, and delivering high-quality enterprise systems.
2 days ago

Project Manager
Location: Johannesburg
Salary:
Drive large scale technology change that truly matters. This role places you at the centre of complex digital transformation where your decisions shape systems, teams, and long term impact. If you thrive in high stakes environments and love turning strategy into reality, this opportunity delivers challenge, influence, and growth.
2 days ago

Full Stack Developer (Entry) 1482
Location: Menlyn
Salary:
Junior Software engineer – AIâEnabled Products (Python)We’re looking for a Junior Software engineer who wants to work at the intersection of software engineering, data, and AIâenabled products.This is not a pure frontend role, not a QA role, and not a research data scientist position.Instead, you’ll be a core engineer helping turn data and machineâlearning capabilities into...
6 days ago

IT Technical Project Manager
Location: Johannesburg
Salary:
Step into a senior delivery role where large scale technology change meets real business impact. This position offers the chance to lead high value initiatives while influencing strategy, stability and long term growth within a complex technology environment.
6 days ago

Business Analyst
Location: Johannesburg
Salary:
Our client is looking for Business Analysts to work alongside software professionals and client stakeholders, playing a key role in defining, analysing, and improving business and system requirements to deliver effective technology solutions.
7 days ago

C# Developers / Architects
Location: Johannesburg
Salary:
Our client is looking for high-calibre C# Developers and Architects to join their consulting delivery teams. This is an opportunity to work on complex, large-scale software solutions across a variety of industries, using modern engineering practices and robust Microsoft technology stacks.
7 days ago

Platform Engineer
Location: Johannesburg
Salary:
Our client is looking for a skilled Platform engineer to join their high-performing team. This is an exciting opportunity to work on modern cloud platforms, building scalable, resilient systems while collaborating with top-tier engineering talent.
7 days ago

Cloud Engineer
Location: Johannesburg
Salary:
We are seeking a specialised Cloud engineer to design, build, secure, and maintain the infrastructure powering our modern data platforms. Unlike a traditional Cloud engineer focused on general application hosting, this role is dedicated to the unique demands of large-scale data workload, spanning high-volume storage, high-throughput networking, and compute-optimised environments for Spark and SQL-...
7 days ago

UX/UI Designer
Location: Johannesburg
Salary:
Our client is looking for a talented UX/UI Designer with strong leadership capabilities and a passion for creating intuitive, engaging, and user-centred digital experiences that delight users.
7 days ago

Senior Accountant
Location: Johannesburg
Salary:
senior AccountantWhere technical excellence meets global opportunity.
8 days ago

Senior Big Data / Hadoop / AWS Data Engineer and Information Team Lead

Similar jobs you might be interested in:

Create a free job alert for Senior Big Data / Hadoop / AWS Data Engineer and Information Team Lead in Johannesburg