Senior Data Engineer

12/2022 – offen

▫ Develop, maintain, and continuously improve data pipelines using platforms like Snowflake, Azure, and Airflow. Optimizing pipeline execution for ingestion, curation, availability, and compliance
▫ Implement data quality controls and monitor data performance to ensure reliability and scalability and troubleshoot data issues and provide solutions.
▫ Collaborate with cross-functional teams, data science, data governance and IT, to identify data needs and ensure data accuracy, completeness, and consistency. Work directly with data custodians, data owners to own data end to end.
▫ Build CI/CD, ELT / ETL pipelines with Python, Azure Data Factory, Airbyte and Fivetran extracting source data from API’s, SAP, SQL-Server, and PostgreSQL.
▫ Develop data model with data vault and Kimball dimensional methodologies.
▫ Use Terraform for provision and manage infrastructure.

IT Service Management (ITSM)

11/2019 – 5/2023

Design and build highly scalable data pipelines using new generation tools and technologies like AWS, GCP, Snowflake, Streamsets, DBT to induct data from various systems.
Optimization and development of analytical platform based on Snowflake, AWS S3, GCP, Jenkins, Docker and DBT.
Develop and operate data pipelines, model data in the warehouse using SQL/dbt and create more complex data products/applications.
Build, manage, and monitor data warehouse projects.
Understanding business needs, identifying the right data sources, developing scalable and reliable data pipelines to ensure a smooth process.
Participating in architectural planning for our current and upcoming data initiatives on a technical level.

Amazon Web Services (AWS), Data Vault, Docker, Git, Java SE, Java (allg.), Jenkins, Microsoft Azure, Microsoft SQL-Server (MS SQL), Microsoft SQL Server Integration Services (SSIS), Mysql, Open Source, Postgresql, Python, SQL

8/2018 – 10/2019

Development of software components & fragments including the necessary technical concepts using OSGi framework, Java & J2EE.
Configuration and integration of software after performing documented checks.
Localization, analysis and removal of software errors.
Support the management with control-relevant information in the technical project management.
Developed java components to store, index, query data in Elasticsearch cluster.
Writing Java components to process data on Spark cluster using RDD, Data Frames & Data Sets.

Apache Spark