A data contract is an explicit, versioned agreement between the team that produces a dataset and the teams that consume it. It turns "please don't rename that column" from a hallway request into something a machine can check. This page explains what goes into a contract, how it is enforced, how schemas evolve without breaking people, and where contracts fit next to data quality tools and catalogs.
In most companies, analytics pipelines read directly from tables owned by application teams. Those teams change their schemas for good product reasons, and nobody tells the downstream dashboard, model, or finance report until it breaks. A data contract fixes the root cause: it makes the dataset an interface with a named owner, the same way a REST API is an interface.
The contract is usually a YAML or JSON file stored in the producer's repository. It describes what the data looks like, what it means, how fresh and complete it will be, and who to call when it is wrong. The producer promises to honour it; consumers build only on what the contract guarantees, never on undocumented columns that happen to exist.
Pipelines rarely fail because of bad code. They fail because an upstream assumption silently changed: a column became nullable, a status enum gained a value, an amount switched from cents to dollars, or a late-arriving batch was skipped. Every one of those is a change in the interface between two teams. Without a contract, the consumer discovers it in production, often days later, and the fix involves backfills and apologetic emails.
Contracts move that discovery to the producer's pull request, where the change is cheap to discuss. They also make ownership explicit, which is the precondition for a data mesh style organisation where domains publish data as a product.
Schema is the structural part: field names, types, nullability, primary keys. It is necessary but not sufficient, because a schema says nothing about meaning. Semantics fill that gap: what one row represents (an order, not a payment attempt), units and currencies, allowed values, and business rules such as "amount is never negative".
Service levels describe delivery: freshness (newest record no older than one hour), completeness, expected volume, and availability. Governance covers the owner, the classification of sensitive fields, retention, and the versioning policy. Open specifications such as the Open Data Contract Standard (ODCS) define a common layout for all of these so tools can read the same file.
A contract is only valuable if something checks it. There are two enforcement points. The first is design time: when a producer opens a pull request, CI compares the proposed schema with the registered contract. Additive changes pass; breaking changes fail unless the producer bumps the major version and agrees a migration window with consumers. Schema registries for Avro or Protobuf on Kafka apply exactly this idea to event streams.
The second is run time: every batch or message is validated before it lands in the trusted zone. Rows that violate the contract go to a quarantine table and raise an alert to the producer, because they own the fix. Consumers then read only validated data, and the checks double as the data quality tests for that dataset.
Contracts use semantic versioning. A useful rule of thumb for which changes are safe:
| Change | Backward compatible? | Version bump |
|---|---|---|
| Add an optional (nullable) column | Yes | Minor |
| Add a value to a documented enum | Usually not; consumers may switch on it | Minor with notice, or major |
| Make a nullable column required | Yes for readers | Minor |
| Make a required column nullable | No | Major |
| Rename or drop a column | No | Major, run old and new in parallel |
| Change type or unit (cents to dollars) | No, and silently dangerous | Major, new column name |
For breaking changes the standard pattern is expand and contract: publish v3 alongside v2, let consumers migrate within an agreed deprecation window, then retire v2. Never change units or meaning in place; add a new field with a new name instead.
The contract below describes an orders dataset. It is deliberately small but contains all four parts.
# orders.contract.yaml -- one file per published dataset
id: urn:contract:sales:orders
version: 2.1.0 # semver: MAJOR = breaking
owner: team-checkout@example.com
description: One row per placed customer order (not per payment attempt).
schema:
- name: order_id
type: string
required: true
unique: true
- name: customer_id
type: string
required: true
pii: false
- name: amount
type: decimal(12,2)
required: true
description: Order total in the order currency, tax included.
checks: [{min: 0}]
- name: currency
type: string
checks: [{allowed_values: [USD, EUR, GBP]}]
- name: created_at
type: timestamp
required: true
sla:
freshness: 1h # newest row no older than 1 hour
completeness: 99.9% # rows delivered vs rows produced
compatibility: backward # consumers on 2.x keep workingA minimal validator reads the contract and checks a pandas DataFrame against it. Production tools (Soda, Great Expectations, dbt model contracts, or the datacontract CLI) do the same thing with more checks and better reporting.
import yaml, pandas as pd
TYPES = {"string": "object", "timestamp": "datetime64[ns]"}
def validate(df: pd.DataFrame, contract_path: str) -> list[str]:
c = yaml.safe_load(open(contract_path))
errors = []
for f in c["schema"]:
col = f["name"]
if col not in df.columns:
if f.get("required"):
errors.append(f"missing required column {col}")
continue
if f.get("required") and df[col].isna().any():
errors.append(f"{col}: {df[col].isna().sum()} nulls")
if f.get("unique") and df[col].duplicated().any():
errors.append(f"{col}: duplicate keys")
for chk in f.get("checks", []):
if "min" in chk and (df[col] < chk["min"]).any():
errors.append(f"{col}: values below {chk['min']}")
if "allowed_values" in chk:
bad = set(df[col].dropna()) - set(chk["allowed_values"])
if bad:
errors.append(f"{col}: unexpected values {sorted(bad)}")
return errors
df = pd.DataFrame({
"order_id": ["o1", "o2", "o2"],
"customer_id": ["c1", "c2", None],
"amount": [19.99, -5.00, 12.00],
"currency": ["USD", "EUR", "JPY"],
"created_at": pd.to_datetime(["2026-10-01"] * 3),
})
for e in validate(df, "orders.contract.yaml"):
print("VIOLATION:", e)Running it reports a duplicate order_id, a null customer_id, a negative amount, and an unexpected JPY currency. In a real pipeline those rows go to quarantine and the producer team gets the alert. Notice that the currency violation is exactly the "enum gained a value" change that breaks downstream reports without anyone noticing.
| Practice | What it guarantees | Who acts | Gap a contract fills |
|---|---|---|---|
| Schema registry (Avro/Protobuf) | Structural compatibility of events | Producer at publish time | No semantics, no SLAs, no ownership |
| Data quality tests | Checks on data already landed | Usually the consumer | Detects breakage after the fact |
| Data catalog | Discovery and documentation | Data stewards | Descriptive only, not enforced |
| dbt model contracts | Column names and types of a dbt model | Analytics engineer | Limited to the warehouse layer |
| Data contract | Schema, meaning, SLAs, owner, versioning | Producer, enforced in CI and runtime | - |