Senior Software Engineer II [Lead Data Engineer / Mumbai OR Bangalore]

At Nielsen, we are passionate about our work to power a better media future for all people by providing powerful insights that drive client decisions and deliver extraordinary results. Our talented, global workforce is dedicated to capturing audience engagement with content - wherever and whenever it’s consumed. Together, we are proudly rooted in our deep legacy as we stand at the forefront of the media revolution. When you join Nielsen, you will join a dynamic team committed to excellence, perseverance, and the ambition to make an impact together. We champion you, because when you succeed, we do too. We enable your best to power our future.

As the Lead Data and AI Engineer, you are the technical architect for Nielsen’s GenAI and data evolution in the region. You will bridge the gap between high-level strategy and functional execution by leading the setup, definition, and industrialization of our Databricks ecosystem. This role is not just about writing code; it is about building a sustainable, governed engineering foundation that transitions experimental AI into enterprise-grade solutions. You will act as the "Technical Commander" for the Databricks environment, ensuring every solution is outcome-driven and architecturally sound.

Core Goals & Responsibilities

• Databricks Ecosystem Leadership: Lead the technical definition and setup of the Databricks environment. You are responsible for ensuring scalability and maintaining the integrity of the Unity Catalog for global data governance.
• End-to-End Engineering: Architect and build robust pipelines using common open-source data formats, ensuring they are optimized for both traditional analytics and emerging AI workloads.
• Operational Excellence: Partner closely with CloudOps and DevOps to ensure 99.99% reliability and seamless CI/CD integration.
• Functional Collaboration: Engage directly with Finance, HR, and Product owners to understand business requirements and translate them into high-impact technical solutions.
• Community & Mentorship: Build and lead a community of developers within the Databricks environment. You will provide technical guidance, unblock delivery friction, and ensure the team builds with "Strategic Foresight" regarding cloud costs (FinOps).

Expertise & Technology Stack

• Core Engineering: 8–10+ years of deep experience in data engineering and architectural design.
• Frameworks & Languages: Expert-level proficiency in Python, Heavy SQL, and dbt for transformation layers.
• AI & Orchestration: Hands-on experience with LangChain and vector database integration for GenAI applications.
• Databricks Mastery: Deep knowledge of Delta Lake, Unity Catalog, and Databricks Workflows.

Analytical Thinking: Ability to challenge status-quo assumptions and provide a pragmatic pivot when technical risks arise.

• Experience: 8–10+ years in a Senior or Lead Engineering capacity.
• Education/Certification: * Required: Proven track record of delivering complex, large-scale data solutions.
• Preferred: Databricks Certified Data Engineer Professional or AWS Certified AI/Machine Learning Specialty.
• Communication: A "Bottom Line Up Front" (BLUF) communication style. You must be able to engage with non-technical stakeholders (HR/Finance) and explain the "how" and "why" of technical decisions clearly.

Please be aware that job-seekers may be at risk of targeting by scammers seeking personal data or money. Nielsen recruiters will only contact you through official job boards, LinkedIn, or email with a nielsen.com domain. Be cautious of any outreach claiming to be from Nielsen via other messaging platforms or personal email addresses. Always verify that email communications come from an @nielsen.com address. If you're unsure about the authenticity of a job offer or communication, please contact Nielsen directly through our official website or verified social media channels.