Deskripsi pekerjaan Senior Data Engineer PT Digital Fajar Buana
Key Responsibilities:
- Productionize Data Science Models
- Build and maintain data pipelines that operationalize data models developed by data scientists for use in production environments.
- Pipeline Development & Containerization
- Develop data processing pipelines primarily in Python, packaged and deployed using Docker containers for portability and reproducibility.
- Workflow Automation
- Design and implement automated workflows to run daily data pipelines, ensuring reliable and timely data processing.
- Data Collection & Maintenance
- Maintain and improve existing data collection programs, troubleshooting and resolving issues as needed.
- Data Integration
- Integrate additional internal and external data sources via APIs, enabling scalable and maintainable data ingestion.
- Reliability & Monitoring
- Implement logging, monitoring, and error-handling mechanisms to ensure robust and maintainable data pipelines.
Required Skills & Experience
- Strong programming experience in Python
- Hands-on experience with Docker and containerized environments
- Experience designing and maintaining data pipelines
- Familiarity with API integration and automated data ingestion
- Experience troubleshooting and maintaining data collection or crawling systems
- Ability to collaborate effectively with data scientists and engineers
Preferred Qualifications
- Experience with AWS services (e.g., EC2, Lambda, ECS/EKS)
- Familiarity with Docker and containerized development environments
- Experience building data integration or ETL pipelines
- Exposure to financial services, luxury goods, or alternative investment markets
- Experience with workflow orchestration tools (e.g., Airflow)
