Data Architect /Data Strategist/ Data Engineer / Data Scientist/ Data & AI Management

11/2021 – 6/2022

Creation of Data Pipeline and Data Flow using Azure Data Factory and Databricks Spark Cluster.

Hands on programming of customized data flow logic in PySpark, Scala, Python to trigger dedicated spark jobs that ensure the end-to-end movement of data from Azure Data Lake through several Postgre Databases to Power BI.

Databricks, Microsoft Azure

8/2021 – 11/2021

Creation of a customized Resnet architecture for image classification. Generally, a wide variety of CNN, RNN and Word-Embedding architectures were explored. Using the aforementioned image features vectors, a muti-stage pipeline of models such as XGBoost, CNN, LSTM, custom Restnet and Unet were assembled.

Data Science, Data Mining, Maschinelles Lernen, Neuronale Netze, Amazon Web Services (AWS)

10/2020 – 7/2021

- Setup predictive maintenance cloud Infrastructure with Azure IoT Hub
- Automate stream data transfer from IoT devices to IoT Hub
- Implementation of Messaging Queue to store IoT data in Azure data lake
- Creation of Databricks notebooks and spark clusters for IoT sensor data analysis - Use PySpark , Deep Neural Networks and other Data Mining and Machine learning techniques to analyse data

Big Data, Data Mining, Internet of Things (IoT)

3/2020 – 2/2021

- Data Hub concept creation and architecture on Azure Cloud
- Implementation of data pipelines for the ingestion in Azure Data Factory.
- Implementation of Azure Functions to pre-process ingested data
- Secure orchestration and movement of data from ADF to Azure Datalake using Azure Key Vault permissions.
- Optimal p partitioning of Datalake data in Databrick Delta Lake format
- Creations of multiple data Apache spark data aggregation scripts in PySpark and Scala
- Setup of Databrick cluster, computation of business KPIs and expose results as rest interface to Qlik, PowerBI visualization

- Setup Azure Active Directory tenant, enable Databricks AD Passthrough, Azure Apps and CosmosDB management.

Apache Spark, Big Data, Microsoft Azure

4/2017 – 1/2019

-Architecture and building of a Machine Learning Python API that is seamlessly integrated to the Spring Boot Java based platform. The Python API uses of Flask/Gunicorn server.
- Build machine learning smart alert system which is based on a combination of self-made outlier detection algorithm and publicly available outlier detection algorithms
- Build of a machine learning system for bank data prediction.
- Using XGBoost and an ensemble of other machine learning techniques for bank data forecasting.
- Deep reinforcement learning for banking network optimization. Usage of both convolution and recurrent neural network to teach multiple agents.

Big Data, Data Mining, Data Science

6/2016 – 7/2019

- Build on-premise Data Centre

- Install on-premise clusters with numerous head and worker nodes using Linux Ubuntu

- Build data Ingestion pipelines to ingest data from file protocols (FTP/SFTP), relational databases (e.g., MySQL, Postgres, Oracle, MSQL Server), document database (e.g., MongoDB), ERP systems (e.g., GreenPlum, SAP), file systems

- Architecture and building of a Data Prep Layer whereby imported data are aggregated using custom Spark function in Scala, SparkSQL, Spark UDF.

- Utilization of Hive and HiveContext to improve the speed of spark data aggregation.

- Performance tuning and optimal data structure setup of MongoDB to deliver KPI results with billions of data points within microsecond

- Build micro-services to access business KPI results from MongoDB and other internal databases.

- Build Java and Spring Boot web-based application

Apache Hadoop, Apache Spark, Apache Tomcat, Data Mining, Data Science, Java (allg.), Spring Framework