Seeking a skilled Data Engineer to design and develop warehouse solutions using Azure Synapse Analytics, ADLS, ADF, Databricks, and Power BI. The role involves building data pipelines, optimizing SQL queries, working with large datasets, and implementing automation using DevOps/CI/CD frameworks. The candidate should have expertise in Azure, AWS, Terraform, ETL, Python, and data lifecycle management while collaborating on architecture frameworks and best practices.
Roles and Responsibilities:
Design and develop warehouse solutions using Azure Synapse Analytics, ADLS, ADF, Databricks, Power BI, Azure Analysis Services
Should be proficient in SSIS, SǪL and Ǫuery optimization.
Should have worked in onshore offshore model managing challenging scenarios.
Expertise in working with large amounts of data (structured and unstructured), building data pipelines for ETL workloads and generate insights utilizing Data Science, Analytics.
Expertise in Azure, AWS cloud services, and DevOps/CI/CD frameworks.
Ability to work with ambiguity and vague requirements and transform them into deliverables.
Good combination of technical and interpersonal skills with strong written and verbal communication; detail-oriented with the ability to work independently.
Drive automation efforts across the data analytics team utilizing Infrastructure as Code (IaC) using Terraform, Configuration Management, and Continuous Integration (CI) / Continuous Delivery (CD) tools such as Jenkins.
Help build define architecture frameworks, best practices C processes. Collaborate on Data warehouse architecture and technical design discussions.
Expertise in Azure Data factory and should be familiar with building pipelines for ETL projects.
Expertise in SǪL knowledge and experience working with relational databases.
Expertise in Python and ETL projects
Experience in data bricks will be of added advantage.
Should have expertise in data life cycle, data ingestion, transformation, data loading, validation, and performance tuning.
Azure Synapse Analytics, ADLS, ADF, Databricks, Power BI, Azure Analysis Services, SSIS, SQL, Query Optimization, ETL, Data Pipelines, Data Science, Analytics, Azure, AWS, DevOps, CI/CD, Terraform, Jenkins, Infrastructure as Code (IaC), Python, Data Ingestion, Transformation, Performance Tuning, Relational Databases, Data Warehousing.