Role Overview

We are seeking a highly skilled and motivated Data Engineer with 5–7 years of experience to design, build, and optimize scalable data pipelines and platforms. The ideal candidate will have strong expertise in SQL, Databricks, Confluent (Kafka), Azure Data Factory (ADF), and Azure Data Lake Storage (ADLS), with a solid understanding of banking domain processes and regulatory considerations. This role involves working closely with cross-functional teams to enable data-driven decision-making across critical banking functions.

Key Responsibilities

• Design, develop, and maintain robust, scalable data pipelines using Azure Data Factory (ADF) and Databricks
• Implement batch and real-time data ingestion frameworks leveraging Confluent Kafka
• Build and optimize data models, transformations, and ETL/ELT pipelines using SQL and Spark
• Develop and manage data storage solutions in Azure Data Lake Storage (ADLS)
• Ensure high data quality, integrity, and governance standards
• Support data integration across systems including core banking and CRM
• Optimize performance of large-scale datasets
• Collaborate with stakeholders to translate requirements into solutions
• Ensure compliance with banking regulations and data privacy
• Troubleshoot production issues

Required Skills & Experience

• Strong expertise in SQL (complex queries, tuning, modeling)
• Hands-on experience with Databricks, Spark, PySpark
• Experience with Confluent Kafka and event-driven architecture
• Experience with Azure Data Factory (ADF)
• Experience with Azure Data Lake Storage (ADLS)
• Understanding of data warehousing concepts
• Familiarity with CI/CD and DevOps for data platforms

Domain Knowledge (Preferred)

• Experience in banking or financial services
• Understanding of core banking systems
• Knowledge of payments, lending, wealth management
• Awareness of regulatory frameworks (KYC, AML)

Soft Skills

• Strong analytical and problem-solving skills
• Excellent communication and stakeholder engagement
• Ability to work in fast-paced environments
• Strong ownership and collaboration mindset

Qualifications

• Bachelor’s or Master’s degree in relevant field
• Azure/Data engineering certifications preferred

Good to Have

• Experience with Python or Scala
• Exposure to data governance tools
• Knowledge of APIs and microservices
• Experience with BI tools like Power BI