Azure Databricks Engineer - Minneapolis, MN
- Location
- Minneapolis, Minnesota, United States
- Salary
- Not specified
- Company
- Akash pawar
Job description
• Role & Company / Project Overview: We are hiring an experienced Azure Databricks Engineer with strong hands-on expertise in Azure cloud data engineering, PySpark, and large-scale ETL/ELT pipeline development. • Key Responsibilities & Daily Duties: - Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark. - Build ETL/ELT solutions to process large volumes of structured and unstructured data. - Integrate data from databases, APIs, data lakes, and streaming platforms. - Develop and optimize Spark jobs for performance, scalability, and reliability. - Implement data orchestration and scheduling using Azure Data Factory (ADF). - Work with Azure Data Lake Storage Gen2 (ADLS Gen2) and cloud data warehouses. - Implement data quality checks, validation, monitoring, and governance processes. - Develop and maintain data models and dimensional models. - Troubleshoot production issues and perform performance tuning. - Follow best practices for CI/CD, cloud security, and data architecture. • Required Qualifications, Skills & Experience: - 10+ years of overall experience in data engineering. - Strong hands-on experience with Azure Databricks. - PySpark, Spark SQL, and Python. - Azure Data Factory (ADF). - Azure Data Lake Storage Gen2 (ADLS Gen2). - Strong SQL development and query optimization. - ETL/ELT pipeline development. - Data warehousing and dimensional modeling. - Delta Lake and Medallion Architecture. - Git, Azure DevOps, and CI/CD pipelines. - Data governance and cloud security best practices. • Working Hours, Shifts & Location Details: - Location: Minneapolis, MN. - Work Model: 100% Onsite. Candidates must be available to work onsite. • Application Instructions: - Interested candidates, please share your updated resume mentioning Azure Databricks Engineer – Minneapolis/Dallas in the subject line.