Job description
This role is for one of the Weekday's clients Salary range: Rs 1500000 - Rs 2500000 (ie INR 15-25 LPA) Min Experience: 4+ years Location: Bengaluru JobType: full-time We are looking for a highly skilled and motivated Big Data Engineer with 4–8 years of experience in designing, developing, and optimizing large-scale data processing systems.
The ideal candidate will have strong expertise in Scala, Spark, and Hadoop ecosystems, along with hands-on experience in building scalable data pipelines and distributed data processing applications.
This role requires a strong analytical mindset, problem-solving ability, and passion for working with high-volume, high-velocity data environments.
As a Big Data Engineer, you will be responsible for developing robust data solutions that support business intelligence, analytics, and data-driven decision-making across the organization.
You will collaborate closely with data architects, data scientists, software engineers, and business stakeholders to deliver efficient and reliable data platforms.
Key Responsibilities Design, develop, and maintain scalable big data solutions using Scala, Apache Spark, and Hadoop technologies.
Build and optimize batch and real-time data processing pipelines for large datasets.
Develop reusable frameworks and components for data ingestion, transformation, validation, and storage.
Work extensively with distributed computing technologies such as HDFS, Yarn, Hive, and MapReduce.
Implement high-performance Spark applications and optimize data workflows for speed and scalability.
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
Monitor, troubleshoot, and enhance existing big data infrastructure and applications.
Ensure data quality, integrity, and consistency across multiple systems and platforms.
Participate in architecture discussions, code reviews, and technical documentation.
Support deployment, automation, and performance tuning activities in production environments.
Stay updated with emerging big data technologies and recommend improvements to the existing ecosystem.
Required SkillsMust-Have Skills Strong hands-on experience in Scala Expertise in Apache Spark Solid understanding of the Hadoop ecosystem Experience in developing distributed data processing applications Strong knowledge of data structures, algorithms, and parallel processing concepts Experience with large-scale ETL and data transformation pipelines Good understanding of performance optimization and troubleshooting in big data environments Good-to-Have Skills Proficiency in Python Experience with Hive , HDFS , Yarn , and MapReduce Exposure to enterprise-level Big Data platforms and cloud-based data ecosystems Familiarity with workflow orchestration and scheduling tools Knowledge of real-time streaming technologies is an added advantage Understanding of DevOps practices and CI/CD pipelines for data engineering Experience & Qualifications 4–8 years of experience in Big Data Engineering or Data Platform Development Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or related field Strong experience in handling structured and unstructured data at scale Experience working in Agile development environments Excellent communication and collaboration skills Preferred Candidate Profile The ideal candidate is a self-driven professional who thrives in fast-paced environments and enjoys solving complex data engineering challenges.
You should possess strong debugging and optimization skills, attention to detail, and the ability to work independently while contributing effectively within a collaborative team.
Candidates with experience in high-volume data environments, enterprise data lakes, or large-scale analytics platforms will be highly preferred.
This job post has been translated by AI and may contain minor differences or errors.
