Key takeaways
- MLOps is the discipline of operating machine-learning systems in production — the ML analogue of DevOps.
- Core practices include data versioning, experiment tracking, model versioning, CI/CD for models, deployment, monitoring, and governance.
- The hardest parts of production AI are usually not the model — they are data pipelines, reproducibility, and drift monitoring.
- Popular MLOps stacks combine experiment tracking (MLflow, Weights & Biases), pipelines (Kubeflow, Airflow, Dagster), model serving (BentoML, Seldon, Triton), and monitoring (Evidently, Arize, Fiddler).
- The cost of poor MLOps shows up as models that work in notebooks but fail in production, rapid model decay, and inability to reproduce past results.
Why MLOps exists
A model that achieves good accuracy on a researcher’s laptop is a starting point, not a solution. To actually serve predictions to users, the model needs to be accessible via an API or batch job, sized to handle traffic, monitored for failures, updated as data changes, and governed so stakeholders can audit its behaviour. Every step involves engineering, and the full chain is too much for most data-science teams to build from scratch.
MLOps emerged around 2018-2020 as the practices borrowed from DevOps were adapted to the ML use case. Like DevOps, it is culture plus tooling — how teams collaborate and what infrastructure they build. Unlike traditional software, ML systems have data as an input, models that decay over time, and outputs that can be biased or unsafe in new ways.
The MLOps pipeline
Data versioning
Models are only as reproducible as their training data. DVC, LakeFS, and delta-lake tables on top of cloud storage give datasets version history comparable to Git for code. Every training run should reference a specific data version so results can be re-created.
Experiment tracking
Which combination of data, code, and hyperparameters produced the best model? Experiment-tracking tools (MLflow, Weights & Biases, Neptune, Comet) log every training run’s metrics, artifacts, and configuration. This is foundational — without it, teams re-discover the same results repeatedly.
Model versioning and registry
Once trained, a model artifact needs a home. Model registries (MLflow Model Registry, SageMaker Model Registry, Vertex AI Model Registry, internal variants) track versions, lineage, approval status, and deployment targets. This is what lets you roll back when a new model breaks production. See our model training guide for the upstream training side.
CI/CD for models
Like code, models should be tested before deployment. A CI pipeline retrains the model on the latest data, evaluates on a held-out test set, compares metrics to the currently deployed version, and blocks promotion if quality regressed. Shadow deployments (new model runs in parallel with old, predictions compared) and canary releases (new model serves a small percentage of traffic) reduce the blast radius of bad models.
Deployment patterns
Real-time inference runs behind an API — FastAPI or Flask wrappers, inference servers like BentoML and Triton, or cloud services like SageMaker and Vertex AI. Batch inference runs on schedule, processing data in bulk. Edge deployment puts the model on mobile or IoT devices, requiring compression and framework support (ONNX Runtime, TensorFlow Lite, CoreML).
Monitoring
Deployed models decay. Data drifts, user behaviour shifts, external events change the input distribution. Monitoring tracks prediction distributions, feature distributions, proxy metrics for accuracy when ground truth is delayed, and user-impact metrics. When drift exceeds thresholds, alerts fire and retraining triggers. See our model monitoring primer for the details.
Governance
For regulated industries — banking, healthcare, insurance — every model deployed needs documented validation, approval workflows, and audit trails. Model cards, fairness reports, and usage logs are mandatory. Lightweight governance is also useful outside regulation: without records, “who built this model and why” becomes an unanswerable question after a year.
Common MLOps anti-patterns
The notebook-to-production chasm
A data scientist builds a working model in a Jupyter notebook. Productionizing it takes months of reimplementation. The symptoms: missing dependencies, manual preprocessing steps, hardcoded paths, no tests. The cure: training pipelines defined as code from the start, with the same data-transformation logic used in training and inference.
Training-serving skew
Features computed during training differ subtly from those computed during inference — slightly different aggregation windows, different null-handling, different time-zone assumptions. The model performs worse in production than on the test set. The cure: a shared feature store (Feast, Tecton) that computes features identically in training and inference contexts.
No retraining cadence
A model is deployed and forgotten. Six months later, accuracy has silently dropped because the world changed. Without scheduled retraining and monitoring, this is undetected until it becomes a business problem. The cure: a retraining cron or trigger, with gates on the CI/CD pipeline to prevent bad retrains from promoting.
Black-box models in regulated settings
Decisions made by opaque models without explanations invite regulatory and litigation risk in financial services, healthcare, and hiring. The cure: favour interpretable models where stakes are high, or pair complex models with interpretability tooling (SHAP, LIME, integrated gradients).
Tooling landscape
The MLOps tooling space is crowded. A representative stack in 2026:
- Experiment tracking: MLflow or Weights & Biases.
- Feature stores: Feast (open source), Tecton (managed), or home-grown on top of a warehouse.
- Pipeline orchestration: Airflow, Dagster, Prefect, or Kubeflow Pipelines.
- Model serving: BentoML, Seldon Core, KServe, or cloud-managed (SageMaker, Vertex AI, Azure ML).
- Monitoring: Evidently, Arize, Fiddler, WhyLabs, or DIY metrics in Datadog / Prometheus.
- Feature engineering & training: PyTorch Lightning, Hugging Face Transformers, plus the bespoke code for your domain.
Build-vs-buy is an ongoing debate. Cloud-managed MLOps stacks (SageMaker, Vertex AI, Databricks) reduce operational load but lock you in. Open-source stacks are flexible but require more engineering.
MLOps for LLMs vs. classical ML
LLMs have shifted MLOps gravity. For fine-tuned models, the classical workflow still applies. For API-consumed LLMs (GPT, Claude, Gemini), the problem changes: prompts are the primary artifact, A/B tests run on prompts rather than models, monitoring tracks LLM-specific metrics (latency, token cost, response quality), and governance covers prompt injection, PII handling, and content safety. A new category of “LLMOps” tooling — LangSmith, Humanloop, PromptLayer, Helicone — has emerged to address this.
How much does MLOps cost?
For small teams, good MLOps can be built with open-source tools on cloud infrastructure for low hundreds of dollars a month plus engineering time. For enterprises, MLOps platforms can cost six or seven figures annually. The real cost is usually engineering — teams of 2-6 engineers dedicated to MLOps infrastructure are common at mid-size companies running AI in production. For industry-scale context, see our ai industry coverage.
Frequently asked questions
Do I need MLOps if I only use pre-trained models through an API?
Less than full MLOps, but not none. Prompt versioning, cost monitoring, evaluation datasets, regression tests, and incident response all still apply — sometimes called LLMOps when applied to LLM APIs. The lighter footprint reflects that you are not training the model; the weight shifts to evaluation and prompt management.
Is MLOps the same as DevOps?
Overlapping but distinct. MLOps inherits DevOps’s version control, CI/CD, containerization, and infrastructure-as-code practices. It adds data versioning, experiment tracking, model-specific validation, and monitoring for drift and fairness. Teams often have MLOps engineers who are DevOps engineers plus ML literacy, or data scientists plus software engineering skills — neither role alone fully covers the surface.
My models are small and stable. Do I really need this?
If the model genuinely does not decay, basic CI/CD and monitoring may suffice. But most “stable” models turn out to decay over months when data changes slowly. The question is not whether to do MLOps but how much — a small team with a handful of models needs simpler processes than a large bank with hundreds. Start with experiment tracking and basic monitoring; add more as pain shows up.
