Senior Data Engineer und Cloud Entwickler

9/2021 – 7/2022

In order to migrate a classical machine learning model to Azure, which aims to predict the vehicle repair rate, I interpreted the Python project into Spark and used Databricks, MLflow and Azure Data Factory to automate the data ingestion and to parallelize the model training. Front-end data visualization uses Grafana. Tools: Azure Data Factory, Databricks, Spark, Delta Lake, Azure Pipelines, Grafana.

Databricks, Big Data, Continuous Delivery, Datenbankentwicklung, Continuous Integration, Microsoft Azure, Python, Scala

2/2020 – 8/2021

Kitchen furniture blueprints sometimes contain errors which should be avoided before being sent to construction. As a
technical team working with a kitchen provider, we received thousands of kitchen plans on a daily basis. I designed a graph model to
represent the furniture of a kitchen, geometric and graph algorithms to learn and look for errors in them, and a pipelined workflow on
Azure to process the data. Kitchen blueprints as data are first fed into the workflow by Kafka in real time, these are then validated
and sent to a Neo4j cluster by an Azure function. After the kitchen blueprints get ingested and analyzed by Neo4j, the result is sent
to the Cosmos DB and returned to the client, in case an error is detected or a warning occurred. Tools: knowledge graph, Kafka,
Neo4j, Azure Functions, AKS, Python and .NET.

C#, Data Science, Apache Kafka, Kubernetes, Microsoft Azure, Python

10/2018 – 1/2020

Astronomical data can be immense and requires a big data solution to process it. Working together with astronomers, I designed a Spark application for an on premise DWH to process the data and read in from and write out to its different layers. The application includes both batch processing to deal with existent data and streaming to deal with new data coming in. I also optimized the Spark application at different levels which has improved its performance remarkably. Tools: Spark 2.x, Kafka, Hadoop, Java.

Apache Hadoop, Apache Spark, Java (allg.), Apache Kafka, Maschinelles Lernen