Data Scientist / Machine Learning Engineer

2/2023 – 6/2024

Deployment von PySpark KI-Anwendungen in Kubernetes und Hadoop

- Deployment, Skalierung und Überwachung von KI-Anwendungen im Kubernetes Cluster (Stackable), sowie im Hadoop Cluster (Cloudera)
- Einbindung von Monitoring Tools (mlflow) zur kontinuierlichen Modellüberwachung (MLOps)
- Skalierung und Logging der Hyperparametersuche von ML-Pipelines mit PySpark und MLFlow - Beschleunigung der bisherigen Suche um 500%.
- Performance- und Ressourcen Optimierung von verschiedenen Machine Learning Apps durch anpassen von Spark Konfigurationen und Datenpartitionierung
- Einführung von modernen GitOps Praktiken für CI/CD im Migrationsprozess

Apache Hadoop, Apache Spark, Docker, Kubernetes, Python

9/2022 – 7/2024

Development of LLM Based AI-Applications in PySpark

- Fine-tuning of foundational LLM (Llama2, Mistral) for generation of text summaries in German. Development utilizing Parameter Efficient Fine Tuning (PEFT) and Low Rank Adaption (LoRA) with Huggingface.
- Segmentation of multi-dimensional customer data by utilizing efficient LLM feature representations - Enhancement of existing clustering approaches by 2x.
- Detection of data anomalies and duplicates through distributed end-to-end Machine Learning pipelines in PySpark, substantially enhancing data quality.
- Time- and cost optimization of GPU resources through dynamic job scheduling on Nvidia A100 and A30 GPUs.

Apache Hadoop, Apache Spark, Generative KI, Large Language Models, Python

1/2021 – 9/2021

Development of ML-Applications for Data Quality Enhancement

- Development of tree based ML classifiers for detection of anomalies in charge point data.
- Creation of business insights regarding data quality utilizing SQL based database queries.

Python, R (Programmiersprache), Scikit-learn, SQL