Multiple esteemed clients - household names in financial services Large transformation projects impacting real change Opportunities for advancement and rolling contracts Roles include a variety of Permanent, Contract and Fixed term contract opportunities with varying timelines for hire over the next 6 months. All of these opportunities require varying skill levels of Azure technologies and Databricks. All opportunities will require atleast permanent residency or citizenship . With some roles requiring eligibility for baseline clearance. Candidates with active baseline clearance will be looked upon favourably. Role types: Data Engineer Senior Data Engineer Data Migration specialist Senior Databricks Specialist Data Architect Data/Solutions Architect Senior Data Architect Key Responsibilities: Data Engineer Responsibilities: Design, build, and maintain scalable data pipelines and ETL processes on Azure and Databricks. Collaborate with cross-functional teams to understand data requirements and deliver optimized solutions. Implement data integration and transformation solutions to support analytics and reporting. Develop and maintain high-performance data models and queries. Monitor, troubleshoot, and improve existing data workflows to ensure efficiency and reliability. Ensure data quality, security, and compliance with organizational and regulatory standards. Data Architecture Responsibilities: Design and oversee the implementation of end-to-end data architecture solutions on Azure and Databricks. Define best practices, guidelines, and standards for data modeling, storage, and integration. Lead the evaluation and selection of appropriate technologies and tools to meet business needs. Collaborate with stakeholders to design solutions that align with strategic goals and performance requirements. Establish governance frameworks to ensure data consistency, accuracy, and security. Mentor and guide engineering teams to adhere to architectural principles and best practices. Skills and Qualifications: Proven experience in designing, building, and managing data solutions on Azure and Databricks. Strong proficiency in Azure services such as Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, and Azure SQL. Expertise in Databricks, including experience with Spark, Delta Lake, and MLflow. Proficiency in programming languages such as Python, Scala, or SQL. Deep understanding of data modeling concepts, ETL/ELT pipelines, and data warehousing principles. Experience with CI/CD pipelines, version control, and infrastructure as code (e.g., Terraform, GitHub). Familiarity with data security, compliance standards and best practices. Strong problem-solving skills, with the ability to work in fast-paced environments and manage competing priorities. Preferred Qualifications: Certifications in Azure (e.g., Azure Data Engineer, Azure Solutions Architect) or Databricks. Understanding of Machine Learning and AI for system and process automation Experience in building batch and real-time data streaming solutions with technologies such as Kafka or Event Hubs. Knowledge of data visualization tools like Power BI or Tableau. Hands-on experience with DevOps practices and cloud cost optimization. Financial services experience desirable Reference number: BBBH242548 Profession:IT, Technology & DigitalArchitects AnalyticsData Engineering IT, Technology & DigitalData Analysts Company: Hudson Australia Date posted: 15th Jan, 2025