Big Data Engineer

Recruiter:

Indsafri

Job Ref:

JHB000263/Nons

Date posted:

Wednesday, June 22, 2022

Location:

Johannesburg, South Africa

JOB SUMMARY:
Our client is looking for Big Data Engineer to join their vibrant team

JOB DESCRIPTION:

Big Data Engineer
Job Description
Big Data Engineer that will work on the collecting, storing, processing, and analysing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Responsibilities

Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
Implementing ETL process {{if importing data from existing data sources is relevant}}
Monitoring performance and advising any necessary infrastructure changes
Defining data retention policies

Skills and Qualifications

Proficiency with Hadoop v2, MapReduce, HDFS
Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
Experience with Spark
Experience with various messaging systems, such as Kafka or RabbitMQ
Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2OGood understanding of Lambda Architecture, along with its advantages and drawbacks
Experience with Cloudera/MapR/Hortonworks
Proficient understanding of distributed computing principles
Management of Hadoop cluster, with all included services
Ability to solve any ongoing issues with operating the cluster
Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
Experience with integration of data from multiple data sources
Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
Knowledge of various ETL techniques and frameworks, such as Flume

NB! This job is now closed. You can apply for other jobs by uploading your CV.

Similar jobs you might be interested in:

QA Engineer
Location: Johannesburg
Salary: Monthly
We are looking for a skilled QA engineer to join an in-house software department supporting a dynamic group of companies, based in Germiston. This is an exciting opportunity to gain exposure to a wide range of software development projects and contribute to delivering high-quality solutions.
2 days ago

DevOps Engineer
Location: Centurion
Salary: 700 000 Annually
Are you a Kubernetes and Rancher expert with a passion for automation, resilience, and high availability? This is your opportunity to join a forward-thinking tech team where innovation meets impact. If you're driven by the challenge of building robust CI/CD pipelines and maintaining mission-critical infrastructure, this role is for you.
3 days ago

Senior Data Engineer
Location: Johannesburg
Salary: Annual Salary
Are you the BEST Senior data engineer??
3 days ago

Data Engineer
Location: Pretoria
Salary: 850 000 Annually
Are you a skilled data engineer ready to build scalable, high-performance data solutions? Join a dynamic team where your expertise in modern data platforms, cloud technologies, and big data processing will directly support business intelligence, analytics, and client reporting. This is your opportunity to work on impactful projects and grow your career in a forward-thinking environment.
4 days ago

Senior Helpdesk Support
Location: Johannesburg
Salary: TBC Annually
Are you a technically curious IT Operations expert who thrives under pressure and enjoys solving complex infrastructure challenges? Step into a senior role where your skills in automation, identity, endpoint, and hybrid networking will drive secure, scalable solutions across a fast-paced environment.
14 days ago

Senior Data Engineer
Location: Centurion
Salary:
Senior data engineer – Lead a High-Impact data engineering Team! Are you a seasoned data engineering professional with a passion for leading technical teams? We are looking for a Senior data engineer to take on a key leadership role in the development and delivery of large-scale data products and services. If you have hands-on experience in data engineering, Telemetry, or Fleet Management in...
18 days ago

AWS Cloud Data Engineer
Location: Pretoria
Salary:
20 days ago

Data Scientist (AI & ML)
Location: Johannesburg
Salary:
Are you ready to shape the future of Generative AI? This is your opportunity to join a visionary team where your mastery in AI, data engineering, and machine learning isn’t just valued, it’s mission-critical. If you’re passionate about building and deploying GenAI solutions that redefine how businesses operate, then this role was built for you.
27 days ago

Data Architect
Location: Johannesburg
Salary: 700000
In search of a skilled data Architect in JHB!
7 days ago

AWS Data Engineer
Location: Johannesburg
Salary:
51 days ago

Big Data Engineer

Similar jobs you might be interested in:

Create a free job alert for Big Data Engineer in Johannesburg