MLOps basics for teams shipping their first model

Shipping a first model needs far less MLOps than vendors suggest and far more than most teams do. The minimum is this: version your data, code and model together; make training reproducible from a single command; register the model you deploy; serve it behind a stable interface; monitor inputs and outputs for drift and quality; and be able to roll back in minutes. Everything else (feature stores, automated retraining, experimentation platforms) can wait until model two.

TL;DR

• Version data, code and model as one unit so any prediction can be traced back to how it was made.
• Make training a single reproducible command with pinned dependencies and a fixed seed.
• Use a simple model registry, package the model as a container, and serve it behind a versioned API.
• Monitor input drift, output distribution and a real quality signal; define retraining triggers up front.
• Name an owner, wire CI/CD and rollback, and skip everything else until the first model has earned its keep.

What do you actually need to version?

Three things, and the mistake is versioning only one of them:

• Code: training scripts, feature logic, inference code. Git handles this.
• Data: the exact training and validation sets, or an immutable snapshot and query that reproduces them. Tools like DVC or lakeFS, or simply dated, read-only object-storage paths, are enough at this stage.
• Model: the trained artefact plus its metrics, hyperparameters, and the commit and data snapshot that produced it.

Tie them together with a run ID. When someone asks in six months why the model made a decision, you should be able to say which version, trained on which data, from which commit. If you cannot, you do not have a production model; you have a file.

How do you make training reproducible?

Reproducibility is not a purity exercise; it is what lets you retrain with confidence when something changes.

• One entry point: make train or python train.py --config configs/v1.yaml, nothing else.
• Pinned dependencies in a lockfile and a container image built from it.
• Fixed random seeds, logged alongside every run.
• Configuration in files, not in notebooks or someone's shell history.
• Every run logs parameters, metrics and artefacts to an experiment tracker (MLflow or Weights & Biases are both fine), so comparisons are a query, not an archaeology dig.

A good test: a colleague clones the repository, runs the one command, and gets the same metrics within noise. If that takes more than an afternoon, the training code has hidden dependencies you will regret.

How should the model be packaged and served?

Register the chosen model in a registry with a stage label (staging, production, archived) and a short note on why it was promoted. Then package it as a container exposing a versioned prediction endpoint with a schema-validated request and response. Keep the feature computation inside the same container or a shared library, so training and serving cannot silently diverge, the classic "training-serving skew" bug.

Serving options in rough order of complexity: a batch job writing predictions to a table; a small FastAPI service; a managed endpoint on your cloud of choice; a Kubernetes deployment with autoscaling. Pick the simplest that meets latency needs. Most first models are batch, and that is fine. Our MLOps and AI infrastructure team spends a surprising amount of time talking teams out of Kubernetes.

What should you monitor, and when do you retrain?

Monitoring is where first models usually go quiet and die. Three layers:

• System health: latency, error rate, throughput. Standard observability.
• Input drift: distribution of each feature compared with the training set. A population stability index or a simple KS test per feature, computed daily, catches most problems.
• Output quality: the hardest and most important. Where ground truth arrives later (a loan repaid, a claim approved, a customer churned), join predictions to outcomes and track precision, recall or error over time. Where it never arrives, sample predictions for human review weekly.

Define retraining triggers before launch, not after the first incident: a quality metric dropping below a threshold, drift beyond a limit, or a fixed cadence such as monthly. Retraining should be the same single command run against a fresh data snapshot, then evaluated against the current production model on a held-out set before promotion. Our data analytics and predictive AI engagements almost always include this loop as a deliverable, because it is what keeps the model useful past month three.

Stage by stage: what is the minimum viable practice?

How do CI/CD, rollback and ownership fit together?

CI for a model repository should run unit tests on feature code, a smoke training run on a small sample, and an evaluation of the candidate against the current production model on a fixed test set. CD promotes only after a human clicks approve; automating that step is a model-two problem. Rollback means the previous model version is still registered, still built, and can be pointed at with a single configuration change. Practise it once before launch.

Ownership matters more than tooling. Someone must be responsible for the model in production: reading the dashboards, triaging alerts, deciding when to retrain. In a typical engagement, we write a one-page runbook (what the model does, how it is served, what the alerts mean, how to roll back, who to call) and hand it over with the model. It is the cheapest MLOps artefact and the one most teams skip. Our AI consulting and strategy work often starts here, with the operating model rather than the model.

Key takeaways

• Version code, data and model together under a run ID; traceability is the foundation of everything else.
• Reproducible training is one command in a pinned container, logged to an experiment tracker.
• Register the model, package it as a container, serve it behind a versioned schema, and keep features shared between training and serving.
• Monitor drift and a real quality signal, and decide retraining triggers before launch.
• Name an owner, wire CI/CD with human-approved promotion and one-command rollback, and defer everything fancier until the first model has proven itself.

FAQ

Do we need a feature store for our first model?

Almost never. A feature store solves consistency and reuse across many models and teams; with one model, a shared feature library used by both training and serving gives you the consistency without the platform. Revisit the question when a second or third model wants the same features, or when online and offline feature values start to disagree.

How often should we retrain?

It depends on how quickly the world your model describes changes, and you should let the monitoring answer that. Set a quality threshold and a drift threshold and retrain when either is crossed, with a fixed fallback cadence, monthly or quarterly, as a safety net. Retraining more often than your evaluation can validate is worse than retraining less, because you cannot tell whether the new model is better.

What is the smallest MLOps stack that works?

Git plus a container registry, an experiment tracker with a built-in model registry such as MLflow, object storage for data snapshots, a small serving service or managed endpoint, and your existing observability tooling for metrics and alerts. That covers versioning, reproducibility, packaging, serving and monitoring with almost nothing new to run. If you want a second opinion on your setup before launch, talk to us.