AI Data Ops Engineer

Job Description Roles & Responsibilities What you'll own: Engineer batch and stream pipelines using Fabric Data Pipelines / Azure Data Factory, Synapse, or Databricks. Implement data quality rules, schema validation, de-duplication, SCD, and reconciliation checks. Operationalize lineage, cataloging, and classifications with Microsoft Purview; enforce RBAC and access patterns. Automate CI/CD via Azure DevOps or GitHub with environment promotion, infrastructure-as-code (Bicep/Terraform), and secrets management via Key Vault. Build feature stores and model-serving data contracts in partnership with MLOps and AI engineering teams. Own reliability: alerts, runbooks, on-call rotation, and cost and performance optimization. Review and finalize vendor-delivered data pipelines and data architecture for AI projects; ensure compliance with client standards for security, performance, and reliability; approve production readiness. Collaborate with delivery partner squads on interface specifications, test data, and delivery checkpoints; support SIT/UAT and production cutover. Define and enforce Data Contracts and SLAs per priority dataset (schema, refresh frequency, quality thresholds, reconciliation checks, and consumer expectations). Own data incident management: classification, RCA, corrective actions, and prevention of recurring nonconformities. Formalize the handshake with AI/ML/DevOps engineers on feature and embedding pipelines, monitoring hooks, and release gates for data-dependent AI deployments. Core skills and tools required: Python and PySpark, SQL, Lakehouse patterns, medallion architecture. Microsoft Purview: catalog, lineage, classifications; data privacy controls and masking. CI/CD with Azure DevOps or GitHub; IaC with Bicep or Terraform; Docker basics. Observability: Kusto/KQL, Azure Monitor, Log Analytics; performance tuning and FinOps. Production incident response and RCA discipline. Desired Candidate Profile What you bring: 6 8 years in data engineering with strong SQL and PySpark and cloud-native data services. Hands-on experience across the Azure data stack: Fabric/Synapse, Data Factory, ADLS Gen2, Delta Lake/Parquet. Bachelor's in Computer Science, Engineering, or equivalent. Required certifications: Microsoft Certified: Azure Data Engineer Associate (DP-203) Preferred certifications: Microsoft Certified: Azure Fundamentals (AZ-900) Databricks Data Engineer Associate or Professional Company Industry RecruitmentPlacement FirmExecutive Search Department / Functional Area IT Software Keywords AI Data Ops Engineer Get real-time job updates only on our App