Objective of the position :-

Describe your Product Mission here / objective of the role.  
At our unit “Data Foundation - Big Data Management” we aim to offer organizations a robust and scalable solution for managing and deriving insights from vast quantities of data. By utilizing Azure's PaaS components, our platform streamlines the deployment and handling of Big Data workloads, empowering our clients to make data-driven decisions and propel business expansion. Our team is accountable for the creation and upkeep of the Big Data platform built on Azure PaaS components EDL (Enterprise Data Lake). We collaborate intensively with stakeholders to comprehend their needs and devise solutions that fulfill their expectations. The team is also in charge of guaranteeing the platform's scalability, dependability, and security, and for staying abreast with the newest technologies and trends in the big data domain

About the Job:

We are seeking a highly motivated and detail-oriented candidate who will be responsible for all data engineering aspects of the data product layer of our Enterprise Datalake, working with Azure and Databricks, maintaining/developing our framework and CI/CD pipelines and processes in an ambitious team.

Main Task:-

• Develop, extend and optimize the data product layer of our Enterprise Datalake with high code standards, tests and suitable documentation to meet both user and internal needs.

  
  
  

• Collaborate with the product owner and architect to understand data requirements and provide suitable solutions.

  
  
  

• Troubleshoot data and software related issues and optimize pipelines for performance and scalability.

  
  
  

• Work with Azure Databricks, Azure Data Factory/Synapse keeping up to date with the latest features and suggest their implementation if beneficial.

  
  
  

• Demonstrate fluent communication skills in English (spoken and written)  

  
  
  

Job Requirements  :-

Technical&ProfessionalKnowledge (mandatory) 

• Experience in Agile way of working with a DevOps mindset

  
  
  

• In-depth know how with Lakehouse concepts, Databricks and Unity Catalog.

  
  
  

• Strong dataengineering and development skills to maintain and extend the data product layer of our datalake

  
  
  

• Strong knowledge in automation & scripting with tools like Github actions, Azure DevOps pipelines, Azure Data Factory, etc.

  
  
  

Technical&ProfessionalKnowledge (additional plus/nice to have skills) 

• Experience with Azure Datafactory/Synapse data flows and pipelines, which are the basis of a legacy application for data products

  
  
  

• Ideally, automating code generation and code management with the help of AItools

  
  
  

• PowerBIas one of the main tools of our users

  
  
  

• Integration knowledge,how metadata can be exchanged with other applications via APIs e.g. with Colibradatacatalogor an inhouse solution for data product definition

  
  
  

• Ideally, backed up by vendor certifications (e.g. Microsoft, Databricks).

  
  
  

• Team player with strong interpersonal, written, and verbal communication skills 

  
  
  

Education 

Bachelor's degree in computer science, Information Technology, Engineering, Business, or related fields.  

Work experience 

Minimum 4-5years related working experienceas a Dataengineer using Azure, Databricks, Python,Azure Data Factory or Synapse