Big Data Engineer

Recruiter:

Indsafri

Job Ref:

JHB000263/Nons

Date posted:

Wednesday, June 22, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
Our client is looking for Big Data Engineer to join their vibrant team

JOB DESCRIPTION:

Big Data Engineer
Job Description
Big Data Engineer that will work on the collecting, storing, processing, and analysing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Responsibilities

Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
Implementing ETL process {{if importing data from existing data sources is relevant}}
Monitoring performance and advising any necessary infrastructure changes
Defining data retention policies

Skills and Qualifications

Proficiency with Hadoop v2, MapReduce, HDFS
Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
Experience with Spark
Experience with various messaging systems, such as Kafka or RabbitMQ
Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2OGood understanding of Lambda Architecture, along with its advantages and drawbacks
Experience with Cloudera/MapR/Hortonworks
Proficient understanding of distributed computing principles
Management of Hadoop cluster, with all included services
Ability to solve any ongoing issues with operating the cluster
Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
Experience with integration of data from multiple data sources
Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
Knowledge of various ETL techniques and frameworks, such as Flume

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

Data Engineer
Location: Johannesburg
Salary: 840000 Annually
Apply here to showcase your data engineer skills! Work with modern tech, big data, and a team that loves clean, scalable solutions.
7 days ago

Data Engineer / Data Scientist – Midrand / Semi-Remote – R600 PH
Location: Pretoria
Salary: 1152000
data engineer / data Scientist – Midrand / Semi-Remote – R600 PH
11 days ago

Intermediate Cloud Data Engineer – Johannesburg – up to R600k per annum
Location: Johannesburg
Salary: 600 000
Intermediate Cloud data engineer – Johannesburg – up to R600k per annum
24 days ago

Data Engineer/ Scientist – Johannesburg – up to R580 per hour
Location: Johannesburg
Salary: 1113600
data engineer/ Scientist – Johannesburg – up to R580 per hour
45 days ago

Intermediate Cloud Data Engineer – Johannesburg – up to R600k per annum
Location: Johannesburg
Salary: 600 000
Intermediate Cloud data engineer – Johannesburg – up to R600k per annum
51 days ago

Mulesoft Specialist Platform Engineer – Johannesburg – R500 PH
Location: Johannesburg
Salary: 960000
Mulesoft Specialist Platform engineer – Johannesburg – R500 PH
53 days ago

AWS Data Engineer – Johannesburg – R530 to R630 per hour
Location: Johannesburg
Salary: 1209600
AWS data engineer – Johannesburg – R530 to R630 per hour
73 days ago

Lead Data Engineer with Python and Typescript – JHB / CPT / Hybrid – R1.8m PA
Location: Johannesburg
Salary: 1 800 000
Lead data engineer with Python and Typescript – JHB / CPT / Hybrid – R1.8m PA
80 days ago

Lead Data Engineer – Johannesburg – R1.2mil up to R1.5mil per annum
Location: Johannesburg
Salary: 1 500 000
Lead data engineer – Johannesburg – R1.2mil up to R1.5mil per annum
80 days ago

AI and Analytics Engineer
Location: Johannesburg
Salary: Datafin1
AI and Analytics engineer
87 days ago

Big Data Engineer

Similar jobs you might be interested in:

Create a free job alert for Big Data Engineer in Johannesburg