Hadoop Product Support Engineer

Job Description: 

• As a product support engineer, you are expected to take ownership of complex customer environments, mentor junior team members, and drive the reliability of Hadoop and Spark-based data processing systems. You will collaborate closely with customer engineering teams to deliver high-throughput Hadoop and Spark applications, solve complex data challenges during migration and upgrades, ensure reliability, and optimise post-migration system performance.
  

Key Responsibilities

• Strong desire to tackle hard technical problems in Hadoop and proven ability to do so with little or no direct daily supervision.
• Troubleshooting: Provide tier-2/3 support for data or performance issues in Hadoop clusters across the entire technical stack.
• Debugging: Conduct deep-dive debugging and optimisation of Hadoop clusters, including NiFi, Impala, and Spark jobs.
• Migration: Lead product support during ODP Hadoop migrations, upgrades, ensuring post-migration stability, addressing upgrades and evolving technical hurdles.
• Optimisation: Design and optimise distributed Hadoop-based applications to ensure low-latency, high-throughput performance for big data workloads.

Candidates should be comfortable supporting operations in a 24/7 environment and be willing to adjust their working hours accordingly.

Technical Qualification

• Experience: 5+ years of hands-on experience working with hadoop environments.
• Technical proficiency in core hadoop services (HDFS, YARN, and Hive/Impala) and good working knowledge of Kafka, NiFi, Ambari, and Cloudera Manager internals.
• Extensive experience in troubleshooting and debugging hadoop components.
• Linux: Advanced skills in configuring, tuning, and troubleshooting Red Hat and Debian-based distributions.
• Good to have proficiency in Python, Bash, or Scala for system automation and performance monitoring.