Job description
Project Role : Data Engineer
Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
Must have skills : Data Engineering
Good to have skills : NA
Minimum 7.5 year(s) of experience is required
Educational Qualification : 15 years full time education
Summary:
We are seeking a highly experienced Assistant Manager / Manger with strong expertise in ETL, Big Data platforms.The ideal candidate will lead the design, development, and operational support of enterprise-scale data pipelines while also enabling metadata management.
Job Title: Assistant Manager/Manger – Data Engineering
Roles & Responsibilities:
1. Data Architecture & Solution Design
Analyze business requirements and source systems to design scalable, reliable ETL and data integration solutions.
Architect end-to-end data pipelines for batch and near real-time ingestion from heterogeneous sources such as RDBMS, APIs, flat files, and streaming platforms.
Define data mappings, transformation rules, aggregation logic, and reconciliation frameworks.
Design incremental load strategies, CDC pipelines, and large-scale data ingestion frameworks.
2. ETL Development & Optimization
Lead development of complex ETL pipelines using Informatica Big Data Management (BDM).
Design scalable data ingestion frameworks into enterprise data lakes and warehouses.
Develop and optimize big data workloads on Cloudera Hadoop ecosystem including:
o HDFS
o Hive
o Spark
o Impala
o Sqoop
Write advanced SQL and PL/SQL for large-scale data transformations.
Implement partitioning, indexing, and parallel processing strategies.
Conduct code reviews and enforce reusable ETL development standards.
Continuously optimize performance for high-volume batch workloads.
Professional & Technical Skills:
8+ Years (Minimum 2+ years in Team Lead / Support role)
. Production Support & Incident Management
Monitor and support ETL workflows developed in Informatica Big Data Management.
Manage batch scheduling using enterprise tools such as:
o AutoSys
o Control-M
o Stonebranch
o Unix Cron
Investigate and resolve:
o Session failures
o Workflow aborts
o Long-running jobs
o Data mismatches and reconciliation issues
Perform root cause analysis (RCA) and implement preventive solutions.
Coordinate with upstream and downstream application teams for issue resolution.
Participate in on-call rotations and L1/L2 production support.
Ensure adherence to SLAs and operational compliance standards.
4. Must-Have Skills
ETL & Data Integration
o Strong expertise in Informatica Big Data Management
o Deep understanding of enterprise ETL design patterns
Big Data & Hadoop Ecosystem
Hands-on experience with:
o Apache Hive
o Apache Spark
o Apache Impala
o Apache Sqoop
Experience working on Cloudera Data Platform
Additional Information:
- The candidate should have minimum 7.5 years of experience in Data Engineering.
- This position is based at our Chennai office.
- A 15 years full time education is required.
This job post has been translated by AI and may contain minor differences or errors.
