(Senior) Data Engineer
We are looking for a (Senior) Data Engineer based in our Malaysia location to join the global Data Science & Engineering team in ROCKWOOL to support the development of our global data platform for factory data collecting datasets from IoT infrastructure from our plants.
This role involves working on a platform built onAzure, leveragingDatabricks,Airflow,Docker,Blob Storage, andMongoDB.
You will address technical challenges, implement new features, streamlineCI/CD pipelines, automate processes, and enhance governance and monitoring.
We are also planning a migration to a new tech stack most likely based onDatabricks, Azure EventHubandKafka, and you may contribute to tasks related to the new platform in the future.
You will join the operational part of a global Data & AI team where you will collaborate with colleagues in Poland and Denmark.
The operational team in Malaysia will contain data scientists, data engineers and ML engineers and there will be a lot of collaboration with other parallel teams in other locations.
Key Requirements
Technical Skills:3-5+ years of experience in data engineering focused on data integration, ETL, and data warehousing,Proficiency inDatabricksandSpark.Experience withAzure servicesand platform.Programming expertise inPython.Familiarity withDevOps/DataOpspractices.Communication:Fluency inEnglish (C1).Personal Attributes:Proactive and self-reliant in task execution.Capable of proposing solutions and effectively collaborating with stakeholders.Big Advantages
Technical Expertise:Experience withMongoDBandNoSQL databases.Knowledge of scheduling tools, especiallyAirflow.Proficiency in containerization usingDocker.Familiarity with advanced Databricks features like Unity Catalog or DLT.Expertise inSQL databases.Experience managing on-premises or cloud-based servers.Understanding of programming structures and algorithms.Version control expertise (Git, preferably GitHub).Advanced understanding ofdata pipelinesanddata architecture(data warehouses, lakes, vaults).Experience withIaCconcepts and tools, preferablyTerraform.Familiarity withLLM tools(e.g., ChatGPT, GitHub Copilot) to enhance productivity.Nice to Have:Experience withdata modeling,Knowledge of additional programming languages (e.g.,C#,Java, Scala),DBT,Snowflake,Kafka.Other Expectations
Familiarity withScrum/Kanbanmethodologies and tools likeJira.Ability to independently gather requirements and collaborate with business stakeholders.Experience working in aninternational team.