Data Scientist

1/2020 – 4/2020

Took care of the Airflow delivery orchestration code for the data products of a consumer internet behaviour monitoring startup.
Unglamorous, complex dag code that had been written by people who had left the company, and didn't have much documentation, but was essential to the top line.
Step by step grokking and refactoring, writing unit tests, fixing issues, generalising code to add flexibility in delivery cadence and post-processing. Performance optimisation to delivery time (including in the orchestration itself).

Apache Spark, Java (allg.), Python

5/2019 – 9/2019

Various bug fixes and improvements to the page view processing implementation of a consumer internet behaviour monitoring startup.
I also implemented a general (postgres) SQL to Parquet exporter in Java.

Apache Spark, Postgresql, Java (allg.), Scala

12/2018 – 4/2019

Porting, unit and integration testing, documentation.
UDF/UDAF/UDTs written in Java by people who had left this consumer internet behaviour and monitoring startup.
Mostly functions dealing with text and URL matching, information extraction from text and URLs and supporting data structures (e.g. tries)

Apache Spark, Java (allg.), Scala

11/2018 – 3/2023

Several iterations of Machine learning models including:
- Loan default risk estimation
- Credit card repayment risk estimation
- Customer churn
- Demand elasticity and deposit rate optimisation
- Card skimming analysis

Using various methodologies including bayesian probabilistic modelling (with PyMC), causal estimation, deep learning with times series (Tensorflow, RNN) and matrix factorisation.

On AWS environment. FastAPI serving, batch or continuous to Redshift.

Maschinelles Lernen, Tensorflow, Amazon Web Services (AWS)

6/2014 – 4/2019

System architecture, full initial implementation, further development, support and handover to a team for further development

Designed and build an analytics backbone continuously taking customer and product data from several loosely integrated proprietary systems to Kafka for streaming analysis, data fusion and population of a Redshift Data Warehouse and Elasticsearch.

Some interesting aspects:
- Hybrid onsite + AWS, with anonymisation before reaching AWS, and extensive encryption everywhere
- Continuous ingestion from proprietary system's Oracle databases using Goldengate
- Streaming fusion of transaction data pieced from multiple sources
- Unified monitoring with Prometheus and Grafana across onsite and AWS. Comprehensive operational dashboard with processing volumes, lags, downtimes and critical resources

Apache Kafka, Oracle Database, Amazon Web Services (AWS), Elasticsearch