Job description for Data Engineer at PT Rapid Teknologi Indonesia
Key Responsibilities:
- Design, develop, and manage scalable ETL/ELT processes for data ingestion and transformation.
- Build and maintain data pipelines and architectures for data collection, transformation, and storage.
- Collaborate with cross-functional teams to understand data requirements and deliver effective data solutions.
- Ensure data quality, integrity, and security across all stages of data processing.
- Optimize and tune data pipelines for performance and scalability.
- Develop and maintain data models, schemas, and metadata.
- Automate manual data processes to improve efficiency and reduce errors.
- Monitor data pipeline performance and troubleshoot issues as needed.
- Stay updated with the latest technologies and best practices in data engineering.
Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, Mathematics, or related field.
- Proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL, SQL Server).
- Strong programming skills in Python, Scala, or Java.
- Experience with ETL/ELT tools and frameworks (e.g., Apache Airflow, Talend, dbt).
- Familiarity with big data technologies like Hadoop, Spark, Kafka is a plus.
- Knowledge of cloud platforms (AWS, GCP, Azure) and cloud-based data services (e.g., Redshift, BigQuery, Snowflake).
- Understanding of data warehousing concepts and best practices.
- Experience with data modeling and schema design.
