Big Data Engineer

Recruiter:

Indsafri

Job Ref:

JHB000263/Nons

Date posted:

Wednesday, June 22, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
Our client is looking for Big Data Engineer to join their vibrant team

JOB DESCRIPTION:

Big Data Engineer
Job Description
Big Data Engineer that will work on the collecting, storing, processing, and analysing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Responsibilities

Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
Implementing ETL process {{if importing data from existing data sources is relevant}}
Monitoring performance and advising any necessary infrastructure changes
Defining data retention policies

Skills and Qualifications

Proficiency with Hadoop v2, MapReduce, HDFS
Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
Experience with Spark
Experience with various messaging systems, such as Kafka or RabbitMQ
Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2OGood understanding of Lambda Architecture, along with its advantages and drawbacks
Experience with Cloudera/MapR/Hortonworks
Proficient understanding of distributed computing principles
Management of Hadoop cluster, with all included services
Ability to solve any ongoing issues with operating the cluster
Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
Experience with integration of data from multiple data sources
Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
Knowledge of various ETL techniques and frameworks, such as Flume

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

DevOps & Infrastructure Lead
Location: Johannesburg
Salary: 70000.00 Monthly
Our client is seeking a strategic DevOps & Infrastructure Lead to guide their infrastructure and Tier 2 Support function through a structured transformation into a fully DevOps-driven capability. This senior leadership role combines operational accountability with strategic oversight to ensure the stability, scalability, and modernisation of all critical systems.
3 days ago

Senior Backend Engineer (Data & Systems)
Location: Centurion
Salary:
Ready to build the engines that power spatial intelligence? ð Tired of building basic CRUD apps? ð Join a specialized boutique tech hub in Gauteng and architect the high-performance data systems that map out the future of logistics and retail across Africa. We need a Senior Backend Polyglot who can master complex ETL pipelines and turn massive datasets into actionable intelligence. If y...
4 days ago

Data Engineer
Location: Johannesburg
Salary: R 1 000 000 Annually
Are you an experienced data engineer ready to take on a high-impact role? Join a forward-thinking organisation in Sandton and help design scalable data solutions that drive business intelligence. If you have GCP and big data expertise, this is your chance to make a difference!
9 days ago

Senior Full Stack Engineer
Location: South Africa
Salary: Market Related Monthly
Role:Our client is looking for a highly experienced and driven Senior Full Stack engineer with a strong background in enterprise application development using .NET and Angular. This role is perfect for someone who thrives in fast-paced environments and is eager to contribute to scalable, high-performance systems that handle large volumes of data. You will play a pivotal role in designing...
14 days ago

Data Engineer
Location: Johannesburg
Salary:
We are looking for a skilled data engineer,to design, build, and maintain scalable data pipelines and platforms that turn raw data into actionable insights. This permanent role is based in Bryanston, Johannesburg, and offers the opportunity to work with modern data technologies in a collaborative, high-impact environment.
16 days ago

Senior BI Developer
Location: Johannesburg
Salary: Annually
Seeking a BI Developer who will be responsible for data analysis, reporting and visualisation, with a focus on the Microsoft Cloud BI Platform.
17 days ago

Intermediate Data Engineer
Location: South Africa
Salary: Annually
Kick off 2026 like the Proteas chasing a record score—strategic, powerful, and unstoppable! We’re looking for a data engineer who can turn chaos into clarity and pipelines into performance. AI is your wingman, generating up to 40% of code, so you can focus on the big plays.
26 days ago

Senior Data Scientist
Location: Johannesburg
Salary:
57 days ago

Senior Data Engineer
Location: Johannesburg
Salary:
58 days ago

Senior Specialist Data Scientist
Location: Johannesburg
Salary: 1 200 000
Mission/ Core purpose of the Job:To determine insights and develop statistical models from large quantities of structured and unstructured data to shape or meet specific business needs and goals.Trainingdata science and advanced analytics.data base modelling and data warehousing principles (SQL)big data tools (Spark, Hive, etc)Strong Programming skills (R, SAS, Python, C++, Java, etc…)Compu...
Today

Big Data Engineer

Similar jobs you might be interested in:

Create a free job alert for Big Data Engineer in Johannesburg