Job description
We are looking for a Senior Data Engineer to design, build, and maintain enterprise-scale data platforms on Azure.
You will own end-to-end data pipelines from raw ingestion to curated, analytics-ready gold layers while driving data quality standards, mentoring junior engineers, and collaborating directly with product and AI teams.
Pipeline Design & Architecture ▸ Architect and manage scalable ETL/ELT pipelines using Medallion Architecture (Bronze → Silver → Gold) ▸ Design ADF pipelines, data models, and schemas optimised for performance and maintainability ▸ Build near real-time and batch ingestion solutions across structured and semi-structured sources Data Governance & Quality ▸ Implement data quality checks, validation frameworks, and lineage tracking across pipeline layers ▸ Enforce security controls, access policies, and compliance requirements on ADLS Gen 2 and Azure SQL ▸ Monitor pipeline SLAs, diagnose performance bottlenecks, and own incident resolution Engineering Excellence ▸ Drive CI/CD adoption for data pipelines using Azure DevOps or equivalent tooling ▸ Define and enforce coding standards, peer-review practices, and documentation norms ▸ Contribute to Al agent development initiatives as a data platform subject-matter expert Collaboration & Mentorship ▸ Partner with data scientists, analysts, and product managers to translate requirements into reliable data products ▸ Mentor junior engineers through code reviews, design discussions, and structured knowledge sharing ▸ Participate in Agile ceremonies and contribute to sprint planning, backlog refinement, and technical estimation EXPERIENCE & REQUIREMENTS Must-Have ▸ 5-6 years of hands-on Data Engineering experience in production environments ▸ Deep expertise in Azure ecosystem — ADLS Gen 2, ADF, Azure SQL Database, and Delta Lake ▸ Strong command of PySpark, Python, and SQL for large-scale data transformation ▸ Proven track record building and operating Medallion Architecture (Bronze/Silver/Gold) on Azure ▸ Experience designing data warehouses and data lake solutions at enterprise scale ▸ Familiarity with CI/CD pipelines, version control (Git), and DevOps practices in a data context Nice-to-Have ▸ Exposure to AWS or GCP in addition to Azure ▸ Experience supporting or building Al/ML data pipelines and feature stores ▸ Knowledge of streaming frameworks — Kafka, Event Hubs, or Spark Structured Streaming ▸ Azure certifications (DP-203, AZ-900 or equivalent)
This job post has been translated by AI and may contain minor differences or errors.
