Databricks Architect | AWS | Lakehouse Platform - Los
- Location
- Los Angeles, California, United States
- Salary
- Not specified
- Company
- Akash pawar
Job description
• Role & Company / Project Overview Persistent Systems is seeking a seasoned Databricks Architect with deep technical expertise in AWS, Lakehouse architecture, distributed data processing, and enterprise data engineering solutions for a long-term position. The architect will act as a hands-on technical leader responsible for designing enterprise-grade data platforms, optimizing system performance and cloud costs, and guiding engineering teams in deploying modern data architectures. • Key Responsibilities & Daily Duties - Design and implement scalable Databricks Lakehouse solutions on AWS - Build and optimize robust ETL/ELT pipelines using PySpark and Delta Lake - Architect low-latency data delivery and serving-layer solutions beyond Spark alone - Handle multi-terabyte structured, semi-structured, and time-series datasets - Optimize compute resources, caching, partitioning, cluster sizing, and cloud costs - Implement streaming and incremental data processing solutions - Establish CI/CD automation, data quality, lineage, monitoring, and governance standards - Lead architecture reviews, proof of concepts (POCs), and technical design discussions - Collaborate with data engineering, ML, BI, and business teams to drive platform adoption • Required Qualifications, Skills & Experience - 12+ years of relevant industry experience in distributed data systems and architecture - Must-Have Technical Skills: - Databricks Lakehouse Architecture on AWS - Apache Spark, PySpark, Spark SQL, and SQL - Delta Lake, Medallion Architecture (Bronze/Silver/Gold) - Databricks Workflows, Jobs, Notebooks, and Delta Live Pipelines - AWS S3, Glue / Glue Catalog, IAM, Lambda, API Gateway, and Kinesis - Data partitioning, caching, Z-Ordering, file compaction, and Spark performance tuning - Unity Catalog, RBAC, data governance, and encryption - CI/CD, GitHub/GitLab, DevOps, and Infrastructure as Code (Terraform preferred) - Preferred Qualifications: - Databricks certifications in Data Engineering or Machine Learning - Experience with MLflow, Feature Store, and MLOps pipelines - Experience in regulated industries such as BFSI or Healthcare - Strong architectural documentation, stakeholder management, and player-coach leadership skills • Working Hours, Shifts & Location Details - Location: Los Angeles, CA - Duration: Long-Term • Compensation & Benefits - Not specified • Application Instructions - Interested candidates should submit an updated resume to Mohammed Jameel, Lead Technical Recruiter at Softech Group.