Senior Cloud Data Engineer (Databricks & Azure)
- Location
- Ontario-wide, Canada
- Salary
- Not specified
- Company
- Akash pawar
Job description
• Role & Company / Project Overview Global Variable is hiring a Senior Cloud Data Engineer (Databricks & Azure) for a client engagement with the Ontario Public Sector. The role entails designing, building, and optimizing enterprise data pipelines and lakehouse platforms, leading large-scale data migrations from Oracle to Azure, Databricks, and Microsoft Fabric, and supporting high-priority provincial initiatives as required. • Key Responsibilities & Daily Duties - Design, build, automate, and optimize complex ETL/ELT pipelines from source to end user. - Develop unified data platforms using Databricks and Unity Catalog. - Build Azure Data Factory pipelines and custom Databricks pipelines with CI/CD through Azure DevOps and Git. - Migrate and integrate very large datasets (OLAP and OLTP) from Oracle to Azure Data Lake, Databricks, and Microsoft Fabric. - Design fact/dimension models, data mappings, data warehouses, data lakes, and lakehouses. - Ingest and provision structured, semi-structured, and unstructured data. - Apply DataOps best practices, including performance monitoring and tuning. • Required Qualifications, Skills & Experience Must Haves (8+ years of experience): - Designing, building, and optimizing complex ETL/ELT processes with Databricks and Delta Live Tables (DLT). - Building unified data platforms with Databricks and Unity Catalog or equivalent. - In-depth experience with relational databases (Oracle, MySQL, SQL Server), relational and dimensional data modelling, query optimization, replication, and administration. - Advanced SQL skills (PL/SQL, T-SQL). - Big data processing with PySpark. - Developing Azure Data Factory pipelines and Databricks pipelines integrated with CI/CD (Azure DevOps, Git). - Data migration and integration from Oracle to Azure Data Lake, Databricks, and Microsoft Fabric. Also Looking For: - Lakehouse design using medallion architecture. - Advanced hands-on experience with Oracle, Azure SQL Server, and Azure Data Factory. - Cloud experience across Azure, Google Cloud, or AWS. - PL/SQL for ETL, along with troubleshooting, performance tuning, failover, and recovery. - Scripting in Python, Unix shell, or Scala. - Experience within the public sector. • Working Hours, Shifts & Location Details - Location: Ontario, Canada. - Work Arrangement: Onsite, 5 days a week, Monday to Friday. - Contract engagement. • Compensation & Benefits - Not specified. • Application Instructions - Send your resume in Word format as soon as possible before closing.