Job description for Data Engineer (Debezium) at Lawencon International
Support development and operations for data pipelines. Mainly CDC ingesting data from source systems to Oracle On-Premises ODS via Debezium, Solace messaging, and Python consumer applications running on Kubernetes.
Core Responsibilities
- Monitor and maintain Debezium CDC connectors capturing database changes
- Troubleshoot and optimize Python consumer applications processing Solace messages
- Troubleshoot and support transformed data into Oracle On-Premises ODS
- Troubleshoot and resolve pipeline failures and performance bottlenecks
- Support Kubernetes deployment, scaling, and health checks for consumer services
- Maintain runbooks and operational documentation
Requirements:
- Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience).
- Minimum 5 years of experience in Data Engineering or Backend Development, including supporting production systems in regulated environments.
- Minimum 2 years of experience in Python development and RDBMS, including data pipeline development, consumer applications, asynchronous programming, error handling, DataFrame processing, SQL query optimization, data loading, and database performance tuning.
- Minimum 1 year of experience with Debezium (CDC connector configuration and source connectors for Oracle, PostgreSQL, and SQL Server) and Kubernetes (deployments, services, ConfigMaps, secrets, resource management, and troubleshooting).
- Strong understanding of Data Engineering concepts, including ETL/ELT pipeline design, optimization, schema evolution, and SQL DML operations.
- Hands-on experience with Docker, Git, CI/CD pipelines (Jenkins, GitLab CI, GitHub Actions, or similar), and monitoring tools such as Prometheus, Grafana, and ELK Stack.
- Strong analytical, problem-solving, debugging, communication, and collaboration skills, with the ability to work independently and effectively across cross-functional teams.
Nice to Have
- Experience with Oracle RDBMS, including PL/SQL and SQL*Loader.
- Knowledge of Solace message broker.
- Experience with AWS cloud data services.
- Experience in performance tuning for high-throughput data pipelines.
- Exposure to data governance, metadata management, and data lineage tools.
Working Environment
- On-premises and hybrid infrastructure
- On-call responsibilities for critical incidents
