Lead Data Engineer

 

Description:

Aadhya Technologies Inc specializes in delivering cutting-edge IT consulting and top-tier recruitment services to help businesses thrive in a rapidly evolving digital world. We focus on quality, agility, and innovation to solve complex tech challenges and build high-performing teams for our clients across various industries.

 

Role: Lead Data (Databricks) Engineer

Location: Toronto, ON or Mississauga, ON

Job Type: Full-time | On-site (Face-to-Face Interview Required)

Experience Required: 8 to 10+ years

 

Job Summary

We are seeking a highly skilled Azure Data Engineer with deep expertise in Databricks to join our data engineering team. The ideal candidate should bring 8 to 10+ years of experience in designing, building, and optimizing large-scale data pipelines. You'll work closely with cross-functional teams to deliver cutting-edge data solutions with a focus on scalability, reliability, governance, and performance.

 

Key Responsibilities

Data Pipeline Development

  • Build and maintain scalable ETL/ELT pipelines using Databricks
  • Use PySpark, Spark, and SQL to transform and process large datasets
  • Integrate data from Azure Blob Storage, ADLS, and other structured/unstructured sources

Collaboration & Analysis

  • Partner with business and BI teams to deliver tailored data solutions
  • Understand data requirements and translate them into efficient technical implementations

Performance & Optimization

  • Optimize Databricks workloads for performance and cost efficiency
  • Proactively monitor and troubleshoot data pipelines

Governance & Security

  • Implement and enforce data access and governance policies using Unity Catalog
  • Ensure compliance with data security and organizational standards

Deployment & CI/CD

  • Use Databricks Asset Bundles for seamless deployment of jobs and notebooks
  • Collaborate using GitLab for code versioning and DevOps automation

 

Technical Skills

  • Strong experience with Databricks (Delta Lake, Unity Catalog, Delta Live Tables, Runtime, etc.)
  • Proficient with Azure Cloud Services (ADLS, ADF, Blob Storage, etc.)
  • Expertise in Spark, PySpark, and SQL
  • Familiarity with Databricks Asset Bundles and version control via GitLab
  • Solid understanding of data architecture and distributed processing frameworks

 

Preferred Experience

  • Working knowledge of Databricks Runtime configurations
  • Experience with real-time streaming tools like Spark Streaming
  • Building real-time or near-real-time data processing pipelines

 

Certifications(Preferred, not mandatory)

  • Azure Data Engineer Associate
  • Databricks Certified Data Engineer Associate

 

Additional Requirements

  • 8 to 10+ years of relevant experience in data engineering
  • Must attend an in-person face-to-face interview in Toronto or Mississauga office

Apply Now
You'll sign in or register, then come back to this job.
No resume yet? Make one free and apply.
Organization Aadhya Technologies Inc
Industry IT / Telecom / Software Jobs
Occupational Category Lead Data Engineer
Job Location Mississauga, ON, Canada
Shift Type Morning
Job Type Full Time
Gender No Preference
Career Level Experienced Professional
Experience 8 Years
Posted at 2025-06-29 6:02 pm
Expires on 2026-12-21