Data Contracts

Overview

A data contract is an API-style agreement about a dataset or event stream. It moves quality responsibility to the producer, who publishes a stable interface rather than leaking internal tables.

How it works

The contract lives in version control next to the producing service and is machine-readable.

id: orders.v2
owner: checkout-team
schema:
  order_id: string, required, unique
  total: decimal(12,2), min 0
  status: enum [placed, paid, shipped, cancelled]
sla: freshness 15m, completeness 99.9%
change_policy: additive only within v2

In CI, the build compares the proposed schema to the published contract and fails on incompatible changes such as removing a field or tightening a type. At runtime, producers serialize with a registry such as Avro or Protobuf schemas, and pipelines run checks on null rates, ranges and freshness. Violations route records to a dead-letter location and alert the owner, not consumers.

Consumer-driven testing complements this: consumers register the fields they use, so producers see impact before changing.

Pitfalls

• Documentation only: a contract not enforced in CI or at ingestion is a wiki page.
• Exposing internal database schemas as the contract couples consumers to implementation. Publish a dedicated interface.
• Ignoring semantics: a column keeps its type while its meaning changes, such as cents becoming dollars. Include units and definitions, and test them.
• No version lifecycle: running many old versions forever. Set deprecation windows.
• Centralizing approval in a committee slows every change.

Key takeaways

• Version, own and enforce the dataset interface.
• Fail the producer build, not the consumer dashboard.
• Cover semantics and freshness, not just types.
• Provide an explicit path for breaking changes.