Deskripsi pekerjaan Data Engineer Insignia
We are looking for a talented Data Engineer to join their team. You will be part of an innovative company that helps brands enhance their market reach through digital transformation solutions. Your role will involve working with large-scale data systems, developing data pipelines, and collaborating with data analysts and scientists to improve data quality.
Responsibilities:
- Design, develop, and maintain end-to-end data pipelines for large-scale data processing.
- Optimize and monitor data workflows to ensure high performance and reliability.
- Collaborate with data scientists to improve data quality and deliver actionable insights.
- Implement best practices in data engineering, ensuring scalability and efficiency.
- Troubleshoot and resolve data-related issues in a timely manner.
Requirements:
- Minimum 2 years of experience as a Data Engineer, Data Analyst Engineer, or similar role.
- Strong proficiency in Python for data processing and automation, especially using Pandas, NumPy, and DuckDB.
- Advanced SQL skills, including experience with:
- Complex queries
- CTAS (Create Table As Select)
- Window Functions
- Common Table Expressions (CTE)
- Query optimization and performance tuning
- Hands-on experience with at least one relational database such as MySQL, PostgreSQL, or MariaDB.
- Experience building, maintaining, and optimizing ETL/ELT pipelines for large-scale datasets.
- Familiar with data cleaning, transformation, and validation processes to ensure data accuracy and consistency.
- Good understanding of data modeling concepts, database schema design, and data warehousing best practices.
- Experience consuming and integrating APIs into data workflows.
- Proficient in development tools and environments such as VS Code, Anaconda, Jupyter Notebook, or similar.
- Familiar with Git/GitHub for version control and collaborative development workflows.
- Exposure to cloud platforms such as AWS, GCP, Azure, or Alicloud is a plus.
- Experience working with BI and visualization tools such as Power BI, Tableau, or Amazon QuickSight is preferred.
- Understanding of workflow orchestration or scheduling tools (e.g., Airflow, Cron Jobs) is an advantage.
- Experience handling real-world business data and production-level implementations/projects.
- Strong analytical thinking and problem-solving skills with attention to detail.
- Able to troubleshoot data-related issues efficiently and provide scalable solutions.
- Good communication skills and ability to collaborate with cross-functional teams including Data Analysts, Data Scientists, and Business Teams.
- Able to work in a fast-paced environment and manage multiple priorities effectively.
- Immediate joiners or candidates with short notice periods are preferred.
