Staff Data Engineer
- Location
- Sydney, New South Wales, Australia
- Salary
- Not specified
- Company
- Akash pawar
Job description
Role & Company / Project Overview: Resource Corner IT is seeking an experienced Staff Data Engineer for a contract opportunity in Sydney, Australia within the Banking domain. The ideal professional is a strong SQL and dbt Core practitioner capable of building reliable, scalable data pipelines using Snowflake, Apache Airflow, Python, and AWS services, with experience handling fixed-format financial data. Key Responsibilities & Daily Duties: • Build, develop, and maintain dbt transformation models across multiple data pipeline layers, from data ingestion to consumer-ready datasets. • Develop and optimise Snowflake SQL transformations to ensure data quality, accuracy, and performance. • Design, develop, and maintain Apache Airflow DAGs for orchestrating data pipelines on AWS. • Develop and maintain AWS Lambda-based data ingestion solutions using Python. • Create, maintain, and update Source-to-Target Mapping (STM) documentation, ensuring clear traceability between source systems and target datasets. • Monitor, troubleshoot, and optimise data pipelines to ensure reliability, scalability, and timely data delivery. • Collaborate with data engineers, analysts, and business stakeholders to understand data requirements and deliver fit-for-purpose datasets. Required Qualifications, Skills & Experience: • SQL: Strong SQL proficiency, including complex queries, data transformations, and query optimisation. • dbt Core: Hands-on experience developing and maintaining dbt models and data transformation workflows. • Apache Airflow: Experience developing and managing DAGs for data pipeline orchestration. • Snowflake: Working experience with Snowflake SQL and data transformation. • AWS Services: Working knowledge of Amazon S3, AWS Lambda, Amazon ECS, AWS Glue Data Catalog, and Amazon EventBridge. • Financial Data Processing: Experience working with fixed-format financial data files. • CI/CD: Experience implementing and maintaining CI/CD pipelines using GitHub Actions. • Apache Iceberg: Knowledge or experience with Apache Iceberg is desirable (nice to have). • Work Authorization: Must hold a valid Australian work visa. Visa sponsorship is not available for this role. Working Hours, Shifts & Location Details: • Location: Sydney, New South Wales, Australia. • Role Type: Contract. Application Instructions: • Interested candidates meeting all requirements should share an updated CV via email.