A model registry is the system of record for trained models. It stores every version of a model together with the information needed to reproduce, compare, approve, deploy and roll it back. This page explains what a registry records, how versioning and promotion work, and shows a complete MLflow example with an automated promotion gate.
Training produces a file: a pickled scikit-learn estimator, a PyTorch state dict, an ONNX graph. On its own that file says nothing about where it came from or whether it is any good. A model registry wraps each file in metadata and gives it a stable identity: a registered model (for example churn-model) with a sequence of immutable versions (1, 2, 3, ...).
The registry is usually backed by an experiment tracker. Common choices are MLflow Model Registry (also the basis of Databricks and Unity Catalog model management), SageMaker Model Registry, Vertex AI Model Registry, and Weights and Biases. Under the hood they all store artifacts in object storage and metadata in a database.
Without a registry, teams end up with folders called model_final_v2_really_final.pkl, and nobody can say which file is serving traffic or which data trained it. When predictions go wrong you cannot reproduce the model, you cannot compare it with the previous one, and rolling back means finding an old file and hoping. Regulated industries also need an audit trail: who approved this model, on what evidence, and when.
A registry gives you three things: reproducibility (every version links to code, data and parameters), governance (approvals and stage transitions are recorded), and safe deployment (serving loads a named alias, so promotion and rollback are metadata changes, not redeploys).
A good model version is a complete recipe. The artifact is the serialized model plus its environment (requirements or a conda file). The code reference is a git commit, and the data reference is a snapshot ID, table version (for example a Delta or Iceberg version number), or dataset hash. Parameters and metrics come from the training run. The signature records input and output schemas so the serving layer can reject malformed requests.
Versions are immutable. If you retrain, you create a new version; you never overwrite an old one. That is what makes rollback trustworthy.
Older registries used fixed stages (None, Staging, Production, Archived). Newer MLflow versions replace stages with aliases: free-form pointers such as champion and challenger that can be moved between versions. Serving code loads models:/churn-model@champion. Promotion is moving the alias; rollback is moving it back. Neither requires rebuilding a container.
Promotion should be gated. Typical gates are: metric better than the current champion on the same holdout set, no regression on important data slices, fairness and latency checks, and for high-risk models a human approval recorded in the registry. A challenger can run in shadow mode or take a small share of traffic before it becomes champion.
import mlflow
from mlflow import MlflowClient
from mlflow.models import infer_signature
from sklearn.datasets import make_classification
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import train_test_split
mlflow.set_tracking_uri("sqlite:///mlflow.db") # local registry for the demo
X, y = make_classification(n_samples=5000, n_features=20, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=0)
with mlflow.start_run() as run:
params = {"max_depth": 3, "n_estimators": 200, "learning_rate": 0.1}
model = GradientBoostingClassifier(**params).fit(X_tr, y_tr)
auc = roc_auc_score(y_te, model.predict_proba(X_te)[:, 1])
mlflow.log_params(params)
mlflow.log_metric("test_auc", auc)
mlflow.sklearn.log_model(
model, "model",
signature=infer_signature(X_te, model.predict(X_te)),
registered_model_name="churn-model", # creates a new version
)
client = MlflowClient()
latest = max(client.search_model_versions("name='churn-model'"),
key=lambda v: int(v.version))
print("registered version", latest.version, "auc", round(auc, 3))
# Gate: promote only if better than the current champion
try:
champ = client.get_model_version_by_alias("churn-model", "champion")
champ_auc = client.get_run(champ.run_id).data.metrics["test_auc"]
except Exception:
champ_auc = 0.0
if auc > champ_auc:
client.set_registered_model_alias("churn-model", "champion", latest.version)
print("promoted to champion")
# Serving side: load by alias, not by number
served = mlflow.pyfunc.load_model("models:/churn-model@champion")
print(served.predict(X_te[:5]))The script trains a model, logs parameters, a metric and a signature, and registers the result as a new version of churn-model. It then compares the new version's test AUC with the current champion and moves the alias only if the new model is better. The final lines show the serving side: it never mentions a version number. Run it twice with different parameters and you will see version 2 created and promoted only if it wins. Install with pip install mlflow scikit-learn.
In a mature setup, nobody clicks "promote" by hand for routine retrains. A scheduled or data-triggered pipeline retrains the model, registers a new version, and runs an evaluation job against a frozen benchmark set. The evaluation job writes its results back to the version as tags or metrics. A policy step then decides: if every gate passes, the version gets the challenger alias and starts receiving shadow traffic; after a soak period with healthy online metrics, a final step moves champion.
The registry also acts as the trigger for deployment. Registry webhooks or events (MLflow and the cloud registries all offer some form of this) can notify a deployment service when an alias moves, so the serving fleet reloads the model. Because the serving layer resolves the alias at load time, the deployment itself is boring, which is the goal.
Two practices keep this safe. First, separate environments: many teams run one registry per environment (dev, staging, prod) or use catalog permissions so that only the CI service account can set the production alias. Second, record why a version was promoted: link the evaluation report, the approving person or job, and the date. When an auditor or an incident review asks why version 7 was serving on a given day, the registry answers it in one query.
Finally, remember that a "model" in production is often more than one artifact. Feature transformations, tokenizers, thresholds and prompt templates for LLM systems all change behaviour. Version them together, either by packaging them into a single pyfunc model or by registering a bundle whose version is pinned as a unit.
| Option | Strengths | Trade-offs |
|---|---|---|
| MLflow (open source) | Free, framework-agnostic, aliases, wide ecosystem | You run the server, auth and storage yourself |
| Databricks / Unity Catalog | MLflow plus catalog permissions and lineage | Tied to the Databricks platform |
| SageMaker Model Registry | Model packages, approval status, CI/CD integration on AWS | AWS-specific concepts |
| Vertex AI Model Registry | Integrated with Vertex endpoints and pipelines | GCP-specific |
| Weights and Biases | Strong experiment UI, artifact lineage graph | Commercial SaaS for teams |
| Git + object storage | Simple, no new service | No promotion workflow, weak metadata |