Data Solution Designer (Databricks) — Middle Park, Port Phillip

We are seeking a talented 10 years Solution Designer with experience in Databricks to join our team. The ideal candidate will have a strong background in designing and implementing data solutions using Databricks platform. This role requires a deep understanding of data engineering principles, and cloud-based analytics. As a Solution Designer, you will play a key role in architecting and implementing scalable, efficient, and high-performance data solutions. Responsibilities : 1. Architectural Design : Design end-to-end data solutions using Databricks platform, considering scalability, performance, security, and reliability requirements. 2. Data Modeling : Define data models, schemas, and structures to support efficient data processing, storage, and retrieval within Databricks environment. 3. Data Ingestion : Implement data ingestion pipelines from various sources such as databases, streaming platforms, and external APIs into Databricks for analytics and processing. 4. Data Transformation : Develop data transformation pipelines using Apache Spark and Databricks notebooks to clean, transform, and enrich raw data into meaningful insights. 5. Data Governance : Implement data governance policies, access controls, and data lineage tracking mechanisms within Databricks to ensure data quality, compliance, and auditability. 6. Performance Optimization : Optimize data processing workflows, queries, and jobs within Databricks to achieve optimal performance and resource utilization. 7. Integration : Integrate Databricks with other data platforms, tools, and services within the data ecosystem to enable seamless data flows and interoperability. 8. Documentation : Document solution designs, data flows, architectural decisions, and best practices for Databricks usage to facilitate knowledge sharing and collaboration within the team. 9. Collaboration : Collaborate with data engineers, data scientists, business analysts, and stakeholders to understand requirements, design solutions, and deliver value-added insights. Qualifications : 1. Bachelor's degree in Computer Science, Information Technology, or related field. 2. Proven experience as a Solution Designer or Data Engineer with expertise in Databricks platform. 3. Deep understanding of big data technologies, distributed computing, and data engineering principles. 4. Hands-on experience with Apache Spark, Scala/Python programming languages, and Databricks notebooks. 5. Familiarity with cloud-based data platforms such as Azure Databricks, AWS EMR 6. Experience in data modeling, ETL/ELT processes, and data integration techniques. 7. Knowledge of data governance, security, and compliance practices in data management. 8. Strong analytical, problem-solving, and communication skills. 9. Ability to work collaboratively in a cross-functional team environment and drive results

Applications close Sunday, 9 June 2024
Take me to the job
Find more jobs nearby: South Yarra, Cremorne, Prahran, Middle Park, Windsor.