Job description for Data Engineer at Indonesia Global Solusindo
Responsibilities:
- Build and maintain large-scale data pipelines using Apache Spark for batch and streaming data processing.
- Develop and manage data flows using Apache NiFi for data orchestration and transformation.
- Implement data integration solutions using Informatica (PowerCenter or IICS).
- Design data schemas and optimize queries for RDBMS (e.g., PostgreSQL, Oracle) and NoSQL (e.g., MongoDB, HBase).
- Contribute to data lake and data warehouse architecture and implementation.
- Proactively monitor and troubleshoot data pipeline issues.
- Collaborate with Data Scientists and Analysts to ensure high-quality data availability.
- Apply best practices in data management, including data governance, security, and privacy.
Qualifications:
- Bachelor’s degree in Computer Science, Information Systems, or a related field.
- Minimum 1 years of experience as a Data Engineer or in a similar role.
- Strong hands-on experience with Apache Spark (RDD, DataFrame, Structured Streaming).
- Proven experience with Apache NiFi for data flow orchestration.
- Hands-on experience in developing and maintaining ETL processes using Informatica.
- Strong SQL skills and deep understanding of databases (PostgreSQL, Oracle, MySQL, etc.).
- Familiarity with Big Data technologies such as Hadoop, Hive, and HDFS.
- Experience with workflow/scheduling tools like Apache Airflow is a plus.
- Understanding of Data Lake, Data Warehouse, and Data Mesh concepts is an advantage.
- Strong communication and teamwork skills.
