Job description for Azure / Microsoft Fabric Data Engineer at The Code Talent Co,. Ltd
Job Title: Azure / Microsoft Fabric Data Engineer
Role Summary
We are seeking an experienced Azure / Microsoft Fabric Data Engineer to design, develop, and maintain modern cloud-based data platforms and analytics solutions. The successful candidate will be responsible for building scalable data architectures, implementing data pipelines, and enabling enterprise analytics capabilities using Microsoft Azure and Microsoft Fabric technologies.
Key Responsibilities
Data Platform Engineering
Design and implement scalable cloud-based data platforms using Microsoft Fabric and Azure services.
Build and maintain enterprise data solutions utilizing OneLake, Lakehouse, Data Factory, Synapse Analytics, and Azure Data Lake Storage.
Implement modern data architectures following Medallion Architecture principles (Bronze, Silver, and Gold layers).
Data Pipeline Development
Develop end-to-end ETL/ELT pipelines and data integration solutions.
Design and maintain orchestration workflows, dataflows, and pipeline frameworks.
Develop notebooks using PySpark and SQL for data transformation and processing.
Automate data ingestion, cleansing, transformation, and loading processes.
Data Management and Optimization
Design and optimize relational and analytical data models.
Perform SQL query optimization and performance tuning.
Monitor and improve data platform performance, reliability, and scalability.
Implement data quality controls and monitoring mechanisms.
Security and Governance
Implement secure data access and secrets management using Azure Key Vault.
Ensure compliance with data governance, security, privacy, and regulatory requirements.
Apply best practices for data protection, auditing, and access control.
DevOps and Collaboration
Support CI/CD processes and deployment automation for data platforms.
Collaborate with business users, analysts, and BI teams to deliver trusted data products.
Document data solutions, technical designs, and operational procedures.
Provide operational support and troubleshooting for production data platforms.
Required Skills and Experience
Technical Skills
Microsoft Fabric (OneLake, Lakehouse, Data Pipelines, Dataflows)
Azure Data Factory (ADF)
Azure Data Lake Storage (ADLS)
Azure Synapse Analytics
ETL/ELT architecture and implementation
Data pipelines and orchestration frameworks
Notebook development using PySpark and SQL
SQL development, optimization, and performance tuning
Medallion Architecture (Bronze/Silver/Gold)
Azure Key Vault
CI/CD and deployment automation
Data governance and security best practices
Preferred Qualifications
Microsoft Azure Data Engineer Associate certification or equivalent.
Microsoft Fabric certification.
Experience with enterprise-scale data warehouse and analytics projects.
Experience with DevOps and Infrastructure-as-Code practices.
Experience Requirements
5+ years of experience in data engineering and cloud data platforms.
Hands-on experience with Azure Data Services and Microsoft Fabric.
Proven experience designing and implementing large-scale data integration and analytics solutions.
Experience working with business intelligence and reporting teams.
