Data & Cloud Consultant

3/2023 – 9/2023

• Conducted data validation and quality checks for the Driving Assistance Project, ensuring accuracy and reliability of collected data.
• Achieved a successful migration of automated-testing project from on premise environment to Microsoft Azure, the optimization of system performance and its scalability.
• Designed the Architecture of the data pipeline for data stream.
• Implemented Kafka for data streaming, facilitating real-time data flow from diverse APIs to the recomputing unit and Test Framework.

Apache Spark, Azure Databricks, Azure Synapse Analytics, Kafka, Docker, Microsoft Azure, Kubernetes

3/2022 – 2/2023

Created and deployed interactive dashboards to monitor employees engagement with Shortways solutions.
Key Achievements:
• Developed interactive AWS QuickSight dashboards for real-time employee engagement monitoring.
• Designed and implemented efficient data pipelines, ensuring seamless data flow.
• Automated ETL processes using AWS Glue (PySpark) to enhance data transformation and loading.
• Managed Data Lake infrastructure to facilitate data storage and retrieval.
• Ingested data from diverse sources using Athena SQL and Python for analysis.
• Streamlined dashboard creation through automation.
Stack: Python, PySpark, AWS (Lambda, S3, RDS, Redshift, Glue, Athena, Quicksight, boto3), docker, git, GitHub, MySQL, TypeScript.

Apache Spark, Microsoft Power BI, mySQL, Docker, Git, Python, TypeScript, Amazon Web Services (AWS)