Azure Data Engineer — The Rocks, Sydney
Expired

Sydney IT & Telecomms DBA / Database Developer Big 4 banking experience highly beneficial Azure / Data Migration/ Pyspark / Data Warehousing Hybrid working environment / Day Rate contract Are you an experienced Data Engineer with a passion for driving large scale data migrations? Do you have a strong background in building data platforms and automated pipelines in Azure? If so, we want to hear from you We’re seeking a Senior Azure Data Engineer to lead the migration of our clients legacy on-premises data warehouse to a cutting-edge data platform on Azure. This is a unique opportunity to work on large data engineering project within the banking sector, delivering solutions that support reporting, analytics, and a variety of downstream systems and third-party data feeds. Key Responsibilities: Lead the migration from on prem data warehouses to Azure based data platforms. Design and build fully automated end-to-end data pipelines in Azure using PySpark and other cloud-native technologies. Collaborate with stakeholders to integrate BI reports, UDAs, downstream systems, and third-party data feeds. Optimise SQL for performance in a cloud data warehouse environment and ensure smooth data flows between systems. Work with both structured and unstructured data types (JSON, XML, AVRO) and handle high-volume batch and near real-time data processing. Contribute to the creation of data products, data mesh architectures, and modern cloud data solutions What We’re Looking For: 7 years of experience in Data Warehousing and Data Engineering within the banking sector. At least 2 years of hands-on experience designing and delivering data engineering solutions with PySpark in Azure. Proven experience in large data migrations and setting up data platforms in the cloud (Azure). Expertise in building automated, scalable data pipelines and advanced SQL for data warehouses, Big Data, and Data Bricks. Proficiency with Big Data tools like Spark (Python/Scala) and experience in cloud data warehouses (Azure Synapse, Data Lake, etc.). Hands-on experience with CI/CD processes using tools such as Bitbucket/GitHub, Jenkins, and Nexus. Experience working with modern architectures, including microservices, REST APIs, and data-as-a-service frameworks. Knowledge of decentralised data architecture – Data Mesh If you feel you meet most of the requirements, I would love to hear from you – if you have any questions, please email kiernan.mcneillmtr.com.au Apply Now

Applications close Sunday, 1 December 2024
Take me to the job
Find more jobs nearby: Sydney, Woolloomooloo, Barangaroo, The Rocks, Haymarket.