Data Engineer, Data Analyst, Python, AWS

12/2016 – offen

I act as an interface between the core statistical / data science world and the engineering world where several innovative ML products are researched and brought live into production. • Design and Implementation of data pipelines for Datalake and collaborate with statisticians / data scientists to ingest and preprocess, enrich data and enable the creation of scientific solutions. • Perform code optimizations and ensure the scalability of the above solutions over different countries and variety and size of different data sources. • Orchestration of the data processing pipelines and invoking the required ML algorithms by spinning of the necessary infrastructure when needed.
• Code review, CI/CD support, enabling monitoring solutions at infrastructure and product level. • Support, design, and implementation of an efficient and integrated working environment for Data Science proof-of-concept work, post research assist in making the solutions production-ready and supporting in post-production operations phase.
• Mentoring and support Junior data engineers and data scientists on the best practices of software engineering and foster knowledge exchange across multiple teams in the client solutions group via various communities of practices.
• Operational support in case of core technical issues of the existing productive solutions.
Technologies: AWS, Glue, Lambda, Python, SQL, Java, Scala, Spark, Docker, Gitlab, Atlassian suite, Terraform, Grafana, Sonarqube, Airflow

Big Data