This is a Data Engineer role with one of the leading companies in AU right now Optiver with an amazing team. They are continuing to grow rapidly. This is the chance to join right as the takes off. More About the Role at Optiver WHAT YOU'LL DO Optiver Sydney is seeking a seasoned Data Engineer to support and advance the data needs of our local research team. You’ll work closely with researchers in Sydney and our Global Research and Data Platform teams to productionise workflows, optimise large-scale data processing, and manage infrastructure costs efficiently. This role requires deep expertise in Data Engineering concepts and technologies, along with a strong ability to work independently and deliver impactful results in a fast-paced environment. You will play a pivotal role in designing and implementing data solutions that enable efficient access to high-quality data, ensuring smooth operations of our data pipelines. Your work will directly support our research teams in deriving real-time insights and making data-driven decisions. As a key data engineering hire in our Sydney office, you will partner with cross-functional teams across regions to enhance and align our data strategy, helping drive Optiver’s data and analytics capabilities forward. Dependent on experience and seniority, key responsibilities would include: - Collaborate with researchers to transform experimental workflows into production-ready data pipelines that meet performance and reliability standards. - Platform and Tooling development: Advancing the start of the art of a variety of Optiver's Research and Data Team's platforms and tools. - Cost Optimisation: Monitor and optimise resource usage in partnership with the Data Platform team, balancing performance and cost-effectiveness. - Monitoring & Tuning: Implement monitoring tools and fine-tune systems for optimal throughput, latency, and reliability, ensuring seamless access to data. - Cross-Functional Collaboration: Work closely with our Data Platform teams globally, maintaining alignment on data strategy, tools, and best practices. - Data Pipeline Design & Development: Architect, build, and manage scalable data pipelines using Spark, Databricks, and proprietary HPC tooling, ensuring high availability, scalability, and performance. - Documentation & Standards: Develop and maintain clear documentation of data pipeline architectures, processes, and best practices. - Mentorship & Guidance: Provide technical mentorship to junior engineers and support the growth of the data engineering function in Sydney. WHO YOU ARE We are looking for a highly skilled Data Engineer with: - 3 years of experience in data engineering. - Demonstrable proficiency in Python - Familiarity with distributed data processing and building robust data pipelines. - Familiarity with the use of or concepts underpinning various Data Engineering technologies and approaches. - Hands-on experience with Spark or Databricks highly desirable. - Ability to lead projects autonomously, prioritise tasks, and deliver high-quality results. - Proven success working with research and platform engineering teams to support data-driven projects. - Strong problem-solving skills, focusing on efficiency, scalability, and cost-effectiveness. - Excellent written and verbal communication skills, with the ability to convey technical concepts to both technical and non-technical stakeholders. - A Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field. - Other preferred qualifications include familiarity with cloud-based architectures (e.g. AWS), large-scale relational databases experience (e.g. PostgreSQL), knowledge of real-time data streaming technologies (e.g. Kafka), and familiarity with system languages (e.g. C++, Rust). WHAT YOU'LL GET You’ll join a culture of collaboration and excellence, where you’ll be surrounded by curious thinkers and creative problem solvers. Driven by a passion for continuous improvement, you’ll thrive in a supportive, high-performing environment alongside talented colleagues, working collectively to tackle some of the most complex problems in the financial markets. In return for your expertise, you will be offered a competitive salary package as well as access to a range of Optiver perks, including: - The chance to work alongside best-in-class professionals - Competitive remuneration, including an attractive bonus structure and additional leave entitlements - Training, mentorship and personal development opportunities - Gym membership plus weekly in-house chair massages - Daily breakfast, lunch and an in-house barista - Regular social events including an annual company trip If you don’t think you're a perfect fit, you should still sign up to Hatch and create a profile, we'll match you to other roles that suit your profile. Hatch exists to level the playing field for people as they discover a career that’s right for them. We model this in our hiring process for our partners like Optiver. ✅ Applying here is the first step in the hiring process for this role at Optiver. We do not discriminate on the basis of gender identity, sexual orientation, cultural identity, disability, age, or any other non-merit factors. To put it simply, Hatch is for everyone.