Data Engineer
Confidential Company
Rekruter Glints membantu mengelola rekrutmen pekerjaan ini.
Dengan melamar ke pekerjaan ini, kamu bisa diproses dan direkrut lebih cepat.
Skills
Deskripsi Pekerjaan Data Engineer
Our client is global leader in solar innovation. Our client is looking for a Data Engineer to design and build databases, Big Data Platforms, Data warehouse, ELT, and ETL pipelines for the Industry 4.0 data program.
The Data Engineer will report to the Senior for the industry 4.0 data program. This person will closely work with Project Lead, Data Analytics Lead, implementation partner, and other key project stakeholders.
The candidate for this role will need to have a meticulous eye for data and appreciates the process of manipulating, storing, and understanding data. This person will closely work with a team of experienced data experts, analysts, and business resources to build databases, data warehouses, and ETL/ELT pipelines as part of the Industry 4.0 data program.
Responsibilities:
- Work with Industry 4.0 data program team which includes project implementation vendor resources, data experts from the manufacturing engineering team, data analysts & business stakeholders.
- Support end-to-end set up of data platform which is scalable and future proof for our client based on the functional/non-functional & technical requirements.
- Integrate data platform with Application from our client and 3rd party system as required from both On-Premise and Multi-Cloud.
- Expand and optimize our data and data pipeline architecture, as well as optimize data flow systems in a secured and scalable manner.
- Assemble large, complex data sets that meet functional / non-functional business requirements, transforming data into formats that are easy to consume and analyze.
- Translate complex business requirements into scalable technical solutions meeting data warehousing design standards.
- Solid understanding of analytics needs and proactiveness to build generic solutions to improve overall efficiency.
- Data Profiling & Cleansing of data by applying data quality checks before ingesting the data.
- Ingest, transform, process, and store data efficiently into a database, data lake, or Datawarehouse platform hosted on-premise or cloud-based environment.
- Design & Build database, Enterprise Datawarehouse design, and data modeling. To be able to organize data at both macro and micro levels and provide logical data models for the consumers.
- Build a data foundation to provide an enterprise view of data across the organization which can be seamlessly accessed by Business users through Self-Service reporting.
- Build a centralized data repository (Sand Box environment) with a robust data foundation to enable data discovery, and data mining for advanced analytics use cases.
- Work with stakeholders including the Product Owner, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Work with data and analytics experts to strive for greater functionality in our data systems.
- Able to communicate effectively, both written and verbal, with technical and non-technical multi-functional teams.
Syarat Pekerjaan Data Engineer
Knowledge and Technical Skills:
- Bachelor's degree in Science or Information Technology
- Certified in Data & Analytics Cloud-based solution
- Proven track record in designing and implementing large-scale data solutions, performing architectural assessments, optimizing data models, establishing and enforcing engineering guidelines and standards on Cloud-based environments
- 4+ years of experience in technical architecture, design & build of Data & Analytics projects in a multinational organization on On-Premise & Cloud-based solutions.
- In-depth experience in working with a data warehouse and knowledge on data warehouse modeling techniques
- Highly knowledgeable in ETL/ELT processes, both design and implementation using various leading tools like for example Informatica, Ab-Initio, Talend, etc
- Experienced in implementing data pipelines for stream and batch ingestion/processing using Open-Source tools (Kafka, Apache Spark) as well as Cloud Native solutions.
- Good experience in building data pipelines that ingests various types of data sources (structured, semi-structured, and unstructured) into a database or data lake and populating a structured warehouse or data mart
- Experienced in the building data lake (distributed storage, Object store), databases & Data warehouse in the cloud (AWS, Azure, GCP), moving data applications to the cloud, and developing cloud-native data applications.
- Experienced in implementing SQL, NoSQL databases & MPP databases
- Experience in Data archiving & Life Cycle Management.
- Data Warehouse design, BI reporting, and Dashboard development.
- Advanced SQL and Python skills are mandatory
- Prior data modeling (dimensional, relational) and data engineering experience is mandatory
- Prior experience in leading a team of technical developers will be mandatory
- A strong understanding of development processes and agile methodologies is necessary
