This is a Senior Data Engineer - PySpark & Data Ingestion role with one of the leading companies in AU right now Westpac with an amazing team. They are continuing to grow rapidly. This is the chance to join right as the takes off. More About the Role at Westpac What’s in it for me? You’ll play a significant part of the future of a business that has been around for over 200 years. Our purpose is to create better futures together. So, we’ll back you in the development of your career, internal career prospects, and flexible working. You’ll also keep learning to grow your career, backed by a fantastic team of people with a can-do, supportive structure. Whatever shape your family takes, we offer generous paid and unpaid parental leave for your nominated primary and support carers. This includes leave to organise adoptions, surrogacy, and foster care arrangements. And we continue to pay your super contributions while you take all the time you need to get your new family settled. What do I need? To be successful in this role you will have the following skills and experience: - Transforming unstructured datasets into a more consumable format using PySpark / Python, but experience with Apache Spark in Java / Scala would also be considered. - Ingesting data from upstream source systems through REST API, File Transfer (SFTP), Bash, and SQL, and working with Network Connectivity, Firewall and Proxy in Linux. - Managing data services (ITSM) in an Agile Scrum environment using Jira and ServiceNow, and improving the operations and CI/CD process for the application and team. - Designing and building data pipelines in a distributed computing environment while tuning performance using techniques like pagination, partitioning, and broadcasting. - Taking ownership in engaging with internal Westpac clients based on data governance and management practices (as 2IC or Tech Lead). Nice to have: - Experience in governance for data onboarding, data quality, master data, and access control. - Experience in the multi-tenancy data lake platforms e.g. Databricks / Snowflake / Cloudera. What’s it like to work there? We are a collaborative team of passionate people with a shared ambition to make a difference for our customers, our communities and each other. At Westpac, making a difference means creating impact, unlocking our own and each other’s passions, and transformative success stories to create better futures together. As well as competitive remuneration and a great culture, joining the Westpac family gives you access to a wide range of employee benefits to help you manage your priorities - whether that means family life, work/life balance, ambition to grow or all the little perks in between. We’ll empower you to shape your career path. Through personalised upskilling, mentoring, and training opportunities, you’re in control of where you start and how you’ll grow. As an equal opportunity employer, we are proud to have created a culture and work environment that values diversity and flexibility – and champions inclusion. If you don’t think you're a perfect fit, you should still sign up to Hatch and create a profile, we'll match you to other roles that suit your profile. Hatch exists to level the playing field for people as they discover a career that’s right for them. We model this in our hiring process for our partners like Westpac. ✅ Applying here is the first step in the hiring process for this role at Westpac. We do not discriminate on the basis of gender identity, sexual orientation, cultural identity, disability, age, or any other non-merit factors. To put it simply, Hatch is for everyone.