DataOps Engineer (m/f/d)
Firmenname für PREMIUM-Mitglieder sichtbar
- November 2024
- Februar 2025
- Bayern
- auf Anfrage
- Remote
- 26.09.2024
- 34614
Projekt Insights
Projektbeschreibung
For our customer in Bavaria we are looking from November for a DataOps Engineer (m/f/d) - 100% remote
Your tasks:
• Architect and develop automated data pipelines on Azure
• Implement data ingestion processes using Azure Data Factory
• Set up and maintain CI/CD pipelines for data workflows using Azure DevOps to automate the testing, integration, and deployment of data pipeline changes
• Optimize ETL/ELT processes for performance and scalability by implementing best practices such as partitioning, indexing, and parallel processing within Azure services
• Implement comprehensive monitoring solutions using New Relic and Azure Monitor to track the performance, health, and failures of data pipelines
• Implement data governance frameworks using Azure Purview to ensure compliance with regulations such as GDPR, CCPA, and HIPAA, and internal policies related to data privacy and protection
• Design and implement Data Vault 2.0 architectures on Azure SQL Database or Azure Synapse Analytics
• Implement and manage the Lighthouse architecture to ensure efficient data access, security, and governance on Azure
• Implement comprehensive data observability solutions using Azure Data Factory, New Relic, and Azure Purview to monitor data quality, lineage, and performance
• Document and communicate data workflows, architectures, and processes
• Provide training and advice project relevant members on data infrastructure and best practices
Your qualifications:
• Proven experience as a DataOps Engineer or similar role, with a focus on automating data pipelines and implementing Data Vault and Lighthouse architectures
• Several years of project relevant experience
• Strong proficiency in programming languages such as Python, SQL, and familiarity with data engineering tools and frameworks on Azure
• Experience with Azure services (Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure Cosmos DB, Azure Monitor, Azure Purview, etc.)
• Experience with relational databases (e.g., Azure SQL Database, Azure Database for PostgreSQL), NoSQL databases (e.g., Azure Cosmos DB), and cloud storage (e.g., Azure Blob Storage)
• Solid understanding of data governance, security, and compliance best practices
Your tasks:
• Architect and develop automated data pipelines on Azure
• Implement data ingestion processes using Azure Data Factory
• Set up and maintain CI/CD pipelines for data workflows using Azure DevOps to automate the testing, integration, and deployment of data pipeline changes
• Optimize ETL/ELT processes for performance and scalability by implementing best practices such as partitioning, indexing, and parallel processing within Azure services
• Implement comprehensive monitoring solutions using New Relic and Azure Monitor to track the performance, health, and failures of data pipelines
• Implement data governance frameworks using Azure Purview to ensure compliance with regulations such as GDPR, CCPA, and HIPAA, and internal policies related to data privacy and protection
• Design and implement Data Vault 2.0 architectures on Azure SQL Database or Azure Synapse Analytics
• Implement and manage the Lighthouse architecture to ensure efficient data access, security, and governance on Azure
• Implement comprehensive data observability solutions using Azure Data Factory, New Relic, and Azure Purview to monitor data quality, lineage, and performance
• Document and communicate data workflows, architectures, and processes
• Provide training and advice project relevant members on data infrastructure and best practices
Your qualifications:
• Proven experience as a DataOps Engineer or similar role, with a focus on automating data pipelines and implementing Data Vault and Lighthouse architectures
• Several years of project relevant experience
• Strong proficiency in programming languages such as Python, SQL, and familiarity with data engineering tools and frameworks on Azure
• Experience with Azure services (Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure Cosmos DB, Azure Monitor, Azure Purview, etc.)
• Experience with relational databases (e.g., Azure SQL Database, Azure Database for PostgreSQL), NoSQL databases (e.g., Azure Cosmos DB), and cloud storage (e.g., Azure Blob Storage)
• Solid understanding of data governance, security, and compliance best practices
Kontaktdaten
Als registriertes Mitglied von freelance.de können Sie sich direkt auf dieses Projekt bewerben.
Kostenlos registrierenÄhnliche Projekte
Top-Auftraggeber
- Verbesserte Wettbewerbssituation | Mit KRONGAARD verbessern Sie Ihre Wettbewerbssituation und sind i
- Renommierte Kunden | Unsere Kunden sind DAX-Konzerne, Banken, Versicherungen und viele weitere bedeu
- Intensive Betreuung | Marketing und Kundengewinnung, Buchhaltung, Vertragsmanagement – das und noch
- Persönlicher Kontakt | In unseren Büros in Hamburg, Berlin, Düsseldorf, Frankfurt am Main und Münche
- Wirtschaftliche Stabilität | Auch wirtschaftlich geben wir Ihnen maximale Sicherheit: Seit Jahren be
Sie suchen Freelancer?
Schreiben Sie Ihr Projekt aus und erhalten Sie noch heute passende Angebote.
Jetzt Projekt erstellen