Azure Databricks Platform Administration
Administer and manage Azure Databricks workspaces, clusters, and jobs
Configure and maintain Databricks environments (Dev, QA, Prod)
Monitor cluster performance, job execution, and system health
Manage notebooks, libraries, and dependencies
Perform upgrades, patches, and platform maintenance
Cluster Management & Optimization
Design and manage cluster configurations (auto-scaling, instance types, pools)
Optimize job performance, resource utilization, and execution times
Troubleshoot cluster failures, memory issues, and job bottlenecks
Implement best practices for Spark performance tuning
Security & Governance
Implement and enforce data security and access controls (RBAC, ACLs, IAM)
Manage workspace security, secrets (Key Vault integration), and credential passthrough
Ensure compliance with enterprise security and governance policies
Implement data governance, auditing, and lineage tracking
Azure Cloud Integration
Integrate Databricks with Azure services such as:
Azure Data Lake Storage (ADLS Gen2)
Azure Key Vault
Azure Data Factory (ADF)
Azure Synapse Analytics
Manage connectivity and networking (VNet, Private Endpoints, NSGs)

✅ Core Skills
10-15 years of IT experience with strong Azure Databricks administration
Hands-on expertise in Apache Spark and Databricks platform management
Strong experience with Azure cloud services (ADLS, ADF, Key Vault, Networking)
Proven experience in production support and platform operations
✅ Technical Skills
Deep knowledge of cluster configuration, job scheduling, and performance tuning
Strong understanding of Spark architecture and optimization techniques
Experience with Terraform / ARM / Bicep (IaC tools)
Knowledge of security models, IAM, RBAC, and data governance
Familiarity with CI/CD tools and DevOps practices
Preferred Skills
Experience with Delta Lake and Lakehouse architecture
Exposure to Python / PySpark / Scala
Familiarity with monitoring tools (Azure Monitor, Splunk)
Knowledge of multi-cloud environments (AWS/GCP)
Experience in data engineering or ETL pipelines