Develop and implement data pipelines for ingesting and collecting data from various sources into a centralized data platform. Develop and maintain ETL jobs using AWS Glue services to process and transform data at scale. Optimize and troubleshoot AWS Glue jobs for performance and reliability. Utilize Python and PySpark to efficiently handle large volumes of data during the ingestion process. Design and implement scalable data processing solutions using PySpark to transform raw data into a structured and usable format. Apply data cleansing, enrichment, and validation techniques to ensure data quality and accuracy. Create and maintain ETL processes using Python and PySpark to move and transform data between different systems. Optimize ETL workflows for performance and efficiency. Collaborate with data architects to design and implement data models that support business requirements. Ensure data structures are optimized for analytics and reporting. Work with distributed computing frameworks, such as Apache Spark, to process and analyze large-scale datasets. Manage and optimize databases, both SQL and NoSQL, to support data storage and retrieval needs. Implement indexing, partitioning, and other database optimization techniques. Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver effective solutions. Work closely with software engineers to integrate data solutions into larger applications. Implement monitoring solutions to track data pipeline performance and proactively identify and address issues Ensure compliance with data privacy regulations and company policies. Stay abreast of industry trends and advancements in data engineering, Python, and PySpark. Requirements Proficiency in Python and PySpark. Strong knowledge of data engineering concepts and best practices. Hands-on experience with AWS Glue and other AWS services. Experience with big data technologies and distributed computing. Familiarity with database management systems (SQL and NoSQL). Understanding of ETL processes and data modeling. Excellent problem-solving and analytical skills. Strong communication and collaboration skills. Bachelor's degree in Computer Science, Information Technology, or a related field.