Job description for Data Engineer at Data Connect Technologies Pte Ltd
Key Responsibilities
Data Engineering
• Design, develop, and maintain robust ETL/ELT pipelines to ingest data from diverse sources (APIs, databases, flat files, SaaS platforms).
• Build and optimize data models, data warehouses, and data lakes to support analytics and reporting needs.
• Ensure data quality, integrity, and availability through monitoring, validation, and automated testing.
• Implement best practices for data governance, security, and compliance (e.g., GDPR, role-based access).
• Optimize SQL queries, stored procedures, and pipeline performance for large-scale datasets.
• Collaborate with analytics and BI teams to deliver clean, well-documented, and reliable datasets.
Power Automate & Workflow Automation
• Design and implement automated workflows using Microsoft Power Automate (Cloud Flows, Desktop Flows, and Process Mining).
• Integrate Power Automate with Microsoft 365, SharePoint, Dataverse, Power BI, Azure services, and third-party APIs.
• Build RPA solutions to automate repetitive manual tasks and reduce operational overhead.
• Develop reusable connectors, custom connectors, and solution packages following ALM best practices.
• Monitor flow performance, troubleshoot failures, and implement error-handling and retry logic.
• Partner with business users to identify automation opportunities and translate requirements into scalable solutions.
Collaboration & Delivery
• Document data pipelines, automation flows, and architectural decisions clearly.
• Participate in code reviews, sprint planning, and Agile ceremonies.
• Mentor junior engineers and contribute to engineering standards and reusable assets.
• Stay current with emerging technologies in the Microsoft data and Power Platform ecosystem.
Required Qualifications
• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field.
• 3+ years of experience as a Data Engineer or in a similar data-focused role.
• Strong proficiency in SQL and at least one programming language (Python preferred; Scala or Java acceptable).
• Hands-on experience building ETL/ELT pipelines using tools such as Azure Data Factory, SSIS, dbt, Databricks, or Apache Airflow.
• Experience with cloud data platforms (Azure Synapse, Snowflake, AWS Redshift, BigQuery, or similar).
• Solid understanding of data modeling concepts (star/snowflake schemas, normalization, dimensional modeling).
• Demonstrated experience designing and deploying Power Automate flows in a production environment.
• Familiarity with the broader Microsoft Power Platform (Power Apps, Power BI, Dataverse).
• Working knowledge of REST APIs, JSON, and authentication patterns (OAuth, service principals).
• Experience with version control (Git) and CI/CD practices.
