Role Summary
As an Azure DevOps Engineer, you will play a pivotal role in designing, implementing, and maintaining our scalable and secure data platform infrastructure. You will leverage your expertise in DevOps, DevSecOps, and DataSecOps to create a robust and efficient environment that supports our data-driven initiatives.
Role Description
- Infrastructure Development: Collaborate with various teams to design and implement a secure and scalable infrastructure platform using AWS/Azure and our Data Center.
- Infrastructure as Code: Utilize Infrastructure as Code (IaC) and containerization to create immutable and reproducible deployments, ensuring scalability and maintainability.
- Performance Optimization: Analyze resource usage to optimize for performance and cost efficiency.
- Data Platform Development: Contribute to the development of our data lake, data warehouse, and stream processing systems, creating a unified query engine, multi-model databases, and analytics tools.
- System Architecture: Design and build systems for high availability, high throughput, data consistency, security, and end-user privacy.
- Mentorship: Mentor other engineers and promote best practices in software engineering, emphasizing monitoring, auditing, reliability, and security.
- Problem-Solving: Develop solutions for scaling data systems to meet various business needs and collaborate in a dynamic and consultative environment.
Core/Must-Have Skills
- Deep understanding of CI/CD tools and a passion for delivering high-quality results.
- Experience with globally distributed log and event processing systems.
- Expertise in DevOps, DevSecOps, and DataSecOps practices.
- Proficiency in managing and scaling container-based infrastructure-as-code technologies.
- Experience designing and building data warehouse, data lake, or lake house solutions.
- Knowledge of data pipeline systems like Dbt, Airflow, Ansible, Argo.
- Expertise in AWS, GCP, or Azure, and associated tools like Terraform, Packer, Docker, Kubernetes, Helm, Prometheus, Grafana, Fluent Bit, Istio.
- Strong background in continuous delivery with Kubernetes and Git.
- Strong end-to-end ownership and a sense of urgency.
- Understanding of cloud storage, compute, and data services.
- Knowledge of cloud network models, security, and connectivity.
- Minimum 6+ years of DevOps experience, including cloud CLI and SDK.
- 6+ years of cloud IaC experience with Terraform/CloudFormation and Ansible/Salt deployment.
- 3+ years of Kubernetes experience focused on DevOps.
- Practical experience with Data Engineering and DevOps/DataOps workflows.
Good-to-Have Skills
- Azure/AWS certifications.
- Experience with Terragrunt and Hashicorp Certified Terraform Associate.
- Familiarity with SDLC and Agile methodologies.