Design and implement ETL processes on GCP, optimize data workflows, ensure data accuracy and integrity, troubleshoot pipeline issues, and collaborate with cross-functional teams to support data integration and analytics for the EDW program.
Responsibilities:
Design, develop, and implement ETL processes on Google Cloud Platform (GCP) to ensure efficient data extraction, transformation, and loading for EDW program.
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions on GCP.
Utilize Google Cloud services, such as native services, BQSQL, DAG, Cloud composer, Dataflow, and Cloud Storage, to optimize data processing and storage workflows.
Create and maintain documentation for ETL processes, ensuring clarity and transparency for future reference and troubleshooting.
Perform data profiling, quality checks, and validation to ensure accuracy and integrity of data throughout the ETL pipeline.
Troubleshoot and resolve issues related to data processing, ensuring timely and accurate delivery of data to end-users.
Stay abreast of GCP updates, emerging technologies, and best practices in ETL development to continually enhance processes.
Collaborate with data engineers, data scientists, and other stakeholders to enhance data integration and analytics capabilities.
Qualifications:
Bachelor’s degree .
Proven experience as an ETL developer, with a focus on Google Cloud Platform, for at least 3-4 years.
Strong programming skills in languages such as Python.
Experience with data modeling, schema design, and database optimization.
Solid understanding of data warehousing concepts and best practices.
Excellent problem-solving and analytical skills.
Strong communication and collaboration skills.
ETL development, GCP, BQSQL, Cloud Composer, Dataflow, Cloud Storage, Python, Data modeling, Schema design, Database optimization, Data warehousing, Collaboration