This role is for one of the Weekday's clients Salary range: Rs 1500000 - Rs 2500000 (ie INR 15-25 LPA) Experience: 7+ yrs Location: Chennai, Coimbatore, Bangalore, Pune Jobtype: full-time We are looking for a highly skilled Lead Data Engineer to design, build, and optimize scalable cloud-based data platforms and processing pipelines for enterprise-grade applications.
This role is ideal for someone who enjoys solving large-scale data engineering challenges and has deep expertise in Google Cloud Platform (GCP), Apache Beam/Dataflow, Java, and BigQuery.
As a Lead Data Engineer, you will play a critical role in building robust batch and real-time data processing systems that enable analytics, reporting, and data-driven decision-making across the organization.
You will work closely with architects, analysts, DevOps teams, and business stakeholders to develop reliable, scalable, and high-performance data solutions.
The ideal candidate is passionate about modern cloud data architectures, distributed systems, and building efficient ETL/ELT pipelines capable of handling large-scale enterprise workloads.
This role requires strong technical expertise, problem-solving ability, and the capability to drive engineering best practices across data platforms.
Key Responsibilities Design and develop scalable ETL/ELT pipelines using Google Cloud Platform services Build and maintain real-time and batch data processing pipelines using Apache Beam and Google Dataflow Develop backend processing components and data transformation services using Java Work extensively with BigQuery for data warehousing, analytics, querying, and performance optimization Integrate data from multiple sources including APIs, relational databases, streaming systems, and cloud platforms Build reliable and scalable streaming data pipelines using technologies such as Kafka and cloud-native services Optimize pipeline performance, scalability, reliability, and cloud infrastructure costs Ensure high standards of data quality, governance, monitoring, security, and operational excellence Collaborate with cross-functional teams including Architects, Analysts, DevOps, and business stakeholders to deliver data-driven solutions Troubleshoot production issues, perform root-cause analysis, and implement long-term scalable fixes Implement CI/CD practices, version control workflows, and automated deployment pipelines for data engineering solutions Design and maintain scalable data models, schemas, and warehouse structures Participate in architecture discussions and contribute to improving overall cloud data platform capabilities Drive best practices in distributed data processing, pipeline optimization, and cloud-native engineering What Makes You a Great Fit Strong hands-on experience with Google Cloud Platform (GCP) services Deep expertise in Apache Beam and Google Dataflow for large-scale data processing Strong programming skills in Java with experience building backend processing systems Hands-on experience with BigQuery and cloud-based data warehousing solutions Experience building and maintaining batch and real-time streaming data pipelines Strong understanding of SQL, data modeling, and distributed data processing concepts Experience working with Kafka or similar streaming technologies Familiarity with CI/CD pipelines, Git workflows, and Agile development methodologies Strong analytical, troubleshooting, and debugging capabilities Understanding of scalable cloud architecture, performance optimization, and reliability engineering Ability to work collaboratively with cross-functional engineering and business teams Experience handling enterprise-scale datasets and complex integration requirements Strong communication skills with the ability to translate technical solutions into business impact Passion for building modern cloud-native data platforms and solving large-scale engineering challenges
