Deskripsi pekerjaan Data Engineer PCS Payment
Job Description:
- Data Pipeline Development: Design, build, and maintain scalable and robust data pipelines (ETL/ELT) to ingest, clean, and transform data from multiple sources.
- Data Integration & Architecture: Manage multi-database synchronization strategies, data migrations, and contribute to the structuring of data warehouses or data lakes.
- Data Quality & Reliability: Ensure data integrity, availability, and security across the data infrastructure. Debug scripts, resolve data conflicts, and troubleshoot pipeline failures.
- Infrastructure Monitoring: Utilize the ELK stack (Elasticsearch, Logstash, Kibana) or similar tools for log and event data management to monitor pipeline health and performance.
- Cross-Functional Collaboration: Work closely with software developers, analysts, and Business Intelligence teams to provide reliable, optimized data that drives strategic business decisions.
Requirements:
- Data Engineering & ETL Savvy: Proven experience building data pipelines and working with orchestration/ETL tools (e.g., Apache Airflow, SSIS, Informatica).
- Expert in SQL & Programming: Extensive experience with complex SQL queries and relational database design. Strong proficiency in programming languages commonly used in data engineering (Python or Golang).
- Hands-On Database Experience: Solid experience working with relational databases such as PostgreSQL, MySQL, and MSSQL Server. Experience with Data Warehouse architecture is highly preferred. Proven experience in designing or building a Data Lakehouse is a strong plus.
- Streaming & Event Processing: Familiarity with message brokers and streaming platforms like Apache Kafka.
- Cloud & Monitoring Tools: Knowledge of the ELK stack. Experience with DBaaS (Database as a Service) on Cloud platforms (AWS, GCP, or Azure) is a strong plus.
- Work Arrangement: Willing to Work From Office (WFO - BSD CITY).
