- A July 2026 analysis by Jervis Anthony Saldanha, published on Medium, argues that production ML failures stem from systemic engineering weaknesses rather than model inaccuracy, with data drift, infrastructure contention and missing observability tooling identified as the primary failure mechanisms.
- According to the Modern Data Blog, MLOps practices that embed monitoring, model registries and CI/CD from the outset can cut deployment timelines from months to days, but most data science projects never reach production at all, blocked by manual workflows and the gap between data science and IT operations.
Most machine learning models that fail in production were not bad models. A July 2026 analysis by Jervis Anthony Saldanha, published on Medium, traces the failures to systems engineering: data drift goes undetected, infrastructure is not sized for concurrent load and no one owns the model once it ships. A separate July 2026 Modern Data Blog report found that most data science projects never reach production at all, blocked by manual workflows and the structural gap between data science and IT operations.
Accuracy Is the Wrong Target
Saldanha’s analysis challenges the assumption that strong validation metrics translate to production reliability. A model that scores well offline can still degrade in live environments through unexpected latency, unhandled edge cases and confidence scores that stay artificially high while actual accuracy drops. The core problem, he argues, is treating deployment as a final-stage task rather than a design constraint that shapes the project from day one. Offline evaluation does not surface the pressures a live system faces.
What Production Actually Demands
Unpredictable bursts of concurrent requests force CPU, memory and network resources to compete in ways sequential low-load testing never reveals. Latency, often treated as an abstraction during development, becomes a direct revenue metric once a model is customer-facing.
Missing data versioning, experiment tracking and model registries compound the problem. Without them, teams lose track of which version is running, who approved it and when it was last retrained. Pascal Janetzky, writing in July 2026 after eight and a half years in ML work, identifies operational discipline as one of the central lessons from that experience. A DevOpsCube July 2026 guide makes the same point about reproducible pipelines and standardised operations. Their persistence as themes across practitioner writing in mid-2026 suggests the gap between development practice and production readiness remains wide.
MLOps as the Fix
MLOps applies software engineering discipline to the ML lifecycle: CI/CD for models, data versioning, experiment tracking, model registries and ongoing monitoring. The shift is from individual model performance to system-level reliability. According to the Modern Data Blog, this can reduce deployment timelines from months to days.
An April 2026 Kernshell guide describes a production-grade model registry as requiring versioned artifacts with comprehensive metadata, including training data lineage and hyperparameters, not just a file store with a version number. For teams assessing where their own pipeline sits, the reasons enterprise GenAI pilots fail to deliver ROI often map directly onto the MLOps gaps Saldanha describes.
Data Drift and Observability
Data drift is the most operationally dangerous failure mode Saldanha identifies. When inference data diverges from training data, model performance declines gradually and silently while dashboard metrics appear stable. Stakeholders see nothing wrong until the damage is visible in business outcomes.
An August 2026 MLflow playbook, “Enterprise AI Adoption Challenges,” lists observability shortfalls as a primary deployment blocker and states that evaluation infrastructure should be a prerequisite for go-live, not a post-launch addition. Tools such as Evidently AI cited in an April 2026 DEV Community article, can track feature drift and trigger alerts when distribution shifts exceed a defined threshold. Real-time monitoring of incoming data distributions and feature integrity, not just model output, is what separates a production-grade system from one waiting to fail quietly.
Pilot-to-Production Gap
An August 2026 MLflow analysis estimates that roughly 30% of generative AI pilots are abandoned after proof-of-concept stage, citing data fragmentation, governance bottlenecks, talent gaps and difficulty proving ROI at scale. A beginner’s roadmap by Sankesh Lal, published August 2026, lists “focusing only on model accuracy” and “avoiding deployment” among the most common mistakes aspiring ML engineers make — a sign the accuracy-first instinct takes hold early and needs to be corrected well before someone reaches production work.
The transition from pilot to production requires budget for integration engineering, ongoing monitoring, retraining cycles and compliance overhead. Teams that do not plan for these costs at the pilot stage rarely find room for them later. Enterprise AI deployment economics bear this out: the integration and governance layer, not the model itself, typically drives cost.
The Skills Gap
Production failures also reflect a talent mismatch. According to an Eventum.ai June 2026 analysis, strong ML engineers in 2026 are expected to demonstrate data discipline, build reproducible pipelines, run comparable experiments and write code designed for production from the start. The challenge for most teams is less about algorithm development and more about operating within complex systems without accumulating technical debt. That requires engineers who can also translate ambiguous business requirements into buildable components, a communication skill, not a modelling one.
MLOps creates the conditions for cross-functional work: data scientists, ML engineers and DevOps teams operating against shared deployment standards. A Coursera December 2025 career roadmap notes that ML engineers, data scientists and MLOps specialists are now distinct roles requiring distinct skill profiles. A team staffed only with data scientists cannot reliably own a production ML system.
Governance and Long-Term Maintenance
Models are not static once deployed. Data distributions shift, business requirements change and regulatory scrutiny around AI is increasing. Without a centralised model registry, production environments accumulate uncertainty about which version is live, who approved it and when it was last retrained.
The MLflow August 2026 playbook recommends a risk-impact matrix that maps each adoption blocker to a named owner and a measurable gate criterion, with evaluation infrastructure and regression test suites treated as deployment prerequisites rather than post-launch additions. Saldanha’s argument, read against the corroborating practitioner literature from mid-2026, is that the ML engineering field already has the modelling capability to build reliable production systems. The missing piece is the operational discipline to treat deployment, monitoring and governance as first-class engineering problems from project kickoff. For teams examining how safe models still fail in multi-agent deployments the systemic framing applies equally: the model is rarely where the failure originates.
