Data Migration Lead/Data Engineer — The Rocks, Sydney
Expired

Join our team as a Data Engineering / Data Migration Lead and take the helm in an exciting migration of data workloads from SAS to Databricks for a major brand client. If you are passionate about Databricks, Spark, and cloud-based data engineering , this is your opportunity to drive a high-impact, transformative project . Immediate start available Job Type: Contract (with potential for full-time permanent position). Apply for a position via our career portal https://ingrity.com/careers/?jobIdWj-MLfcyg9ie About INGRITY: INGRITY is a Microsoft Data and AI Solution Partner dedicated to delivering transformative data and AI-driven solutions. We collaborate with some of the best ASX-listed companies and medium-sized businesses, working closely with Microsoft to create measurable value for our customers. Our success is built on innovation, customer advocacy, and a strategic partnership with Microsoft. About the job: We are seeking a skilled and experienced Data Engineering/ Data Migration Lead with extensive Databricks expertise to lead the migration of data workloads from SAS to Databricks . The ideal candidate will possess a deep understanding of data engineering principles and a strong grasp of data processing, cloud-based data platforms, and ETL optimisation within Databricks. This role requires collaboration with cross-functional teams to ensure a seamless transition, optimise data pipelines, and enhance our data infrastructure. This position is available to start immediately. Key Responsibilities: Lead the migration of data processing and analytics workloads from SAS to Databricks , ensuring a seamless transition with minimal business disruption. Re-engineer SAS-based transformations and business logic into Databricks using Spark SQL, PySpark, or Scala . Design, develop, and optimise data pipelines and ETL workflows in Databricks for scalability and performance. Work closely with data scientists, analysts, and business stakeholders to align data infrastructure with business needs. Implement best practices for data governance, security, and quality assurance within Databricks. Troubleshoot and resolve migration challenges , ensuring data integrity and operational efficiency. Provide technical leadership and mentorship to the team. Document migration strategies, workflows, and processes for future reference. Requirements: 5 years of experience in leading Data Engineering/Data Migration projects, with a strong emphasis on Databricks . Proven expertise in Databricks , including cluster management, Delta Lake, and performance optimization . Strong hands-on experience in Spark (PySpark, Scala, or SQL) for data processing. Deep knowledge of ETL processes, data modelling, and data warehousing concepts . Experience with cloud platforms (Azure, AWS) and their data services , particularly in relation to Databricks . Excellent problem-solving and troubleshooting skills. Strong communication skills , with the ability to explain technical concepts to non-technical stakeholders. Apply for a position through our company career portal using the link below for a quick and easy application process. https://ingrity.com/careers/?jobIdWj-MLfcyg9ie Additional Information: Eligibility: Candidates must have full working rights in Australia to be considered for this role. About INGRITY: We are a fast-growing, progressive Sydney-based data & analytics company. We service many large corporates and SMBs in the financial services sector, to derive value from data. At Ingrity, we are committed to building a diverse and inclusive workforce to deliver value to customers. As a value-driven organisation, we nurture and support our people by having a laser-sharp focus on skill and talent development, collaboration, and flexibility.

Applications close Sunday, 9 March 2025
Take me to the job
Find more jobs nearby: Sydney, Woolloomooloo, Barangaroo, The Rocks, Haymarket.