Senior Databricks Engineer

Senior Databricks Engineer - Long Term Mission - Full Remote

Avanda is a Swiss IT Consultancy. We are currently looking for a Senior Databricks Engineer to join our international team at our global end client. This is a long-term opportunity. You can work remotely with a few business trips to Zurich, CH.

THE ROLE

You will join a B2B Sales Data Aggregation programme built on Databricks. The goal is to unify CRM data from Salesforce, Microsoft Dynamics 365, and Excel/SharePoint into a single harmonised platform and deliver high-quality data products and KPI dashboards for senior leadership.

Your Responsibilities

• Design and build ETL pipelines and CDC/incremental connectors for Salesforce & Dynamics 365 using Databricks-native patterns (Auto Loader / Structured Streaming / Jobs)
• Implement Bronze → Silver transformations using Delta Lake and reusable SQL/Notebook templates
• Develop PySpark notebooks and apply Delta Lake optimisations (compaction, Z-order, caching)
• Build automated data-quality checks and deliver KPI dashboards for C-level stakeholders
• Set up monitoring, alerting, and job orchestration (Databricks Jobs / Observability)
• Document architecture and data lineage (Unity Catalog / Delta time-travel)

Requirements

• 5+ years of data engineering experience
• 3+ years with Databricks and Delta Lake
• Strong PySpark / Python and SQL skills
• Proven experience integrating Salesforce & Dynamics 365 (APIs, CDC, Bulk API)
• Solid background in data modelling, ETL design, and incremental/CDC approaches
• Experience with Azure cloud services and Databricks workspace administration
• Experience building data quality frameworks and monitoring
• English mandatory; German is a plus