Senior Data Engineer

9/2022 – 1/2024

Technologies: Python, REST API, Kafka, Spark, PostgreSQL, Redis, CI/CD, AWS, Grafana, Kubernetes, and ArgoCD.
Roles:
Led the development of an API-driven service that seamlessly handled 17 million monthly page views, achieving an exceptional cache efficiency rate of 94%.
Collaborated closely with the DevOps team to architect and operate large-scale applications on cloud platforms, leveraging cutting-edge technologies such as Kubernetes, Helm Charts, Argo CD, Grafana, and Lambda architecture.
Designed and implemented a highly efficient and normalized relational database, effectively managing data for millions of users and ensuring optimal performance.
Spearheaded a major ETL architecture refactor, resulting in a remarkable efficiency improvement of approximately 50% and significantly reducing processing time.
Utilized Databricks SQL/PySpark for advanced data processing and successfully orchestrated complex workflows using Step Functions, streamlining data operations and enhancing productivity.
Played a key role in enhancing and maintaining robust CI/CD pipelines, enforcing code quality, and enabling seamless and rapid software delivery.
Implemented comprehensive unit and integration tests to ensure superior code quality, high test coverage, and flawless functionality.
Mentored and guided a team of 5 junior developers, sharing valuable knowledge and expertise to accelerate their technical growth and ensure the delivery of high-quality software solutions.

Amazon Web Services (AWS), Django, Docker, Kubernetes, Postgresql, Python, Relationale Datenbanken, SQL, Test Automation

1/2018 – 8/2022

Increased API performance by 300% through successful migration from Relational Database to NoSQL, supporting 7 million monthly requests.
Improved Schufa scoring accuracy by implementing AI techniques, leveraging real-world credit data for over 65 million individuals.
Utilized artificial intelligence techniques to improve the scoring system for Schufa.
Implemented NLP algorithms for credit data normalization and classification, resulting in more accurate and reliable scoring.
Developed automation scripts for ad hoc projects that reduced manual effort by 60% and improved productivity. Demonstrated expertise in working with Big Data Frameworks like Apache Spark to analyze and process massive datasets efficiently.

Apache Spark, Apache Kafka, Mongodb, Postgresql, Python

1/2017 – 12/2022

Collaborated with a team of developers to design and implement web applications using Python, Flask, and Django frameworks.
Utilized Django to build RESTful APIs for seamless communication between front-end and back-end systems. Integrated NLP transformers into projects to process and analyze natural language data for sentiment analysis, text classification, and named entity recognition.
Implemented deep learning models using TensorFlow and PyTorch to solve complex NLP tasks, such as machine translation and text generation.
Developed and maintained database schemas using SQL and ORM tools.
Participated in code reviews, bug fixing, and testing to ensure software quality and reliability.
Actively participated in Agile development sprints, attending daily stand-up meetings and contributing to sprint planning and retrospective sessions.

Django, Python, SQL, Tensorflow, Torch