Data contracts for agents: what they miss and how to test meaning
TL;DR. A data contract is a written, checked agreement between the people who produce or own a dataset and the people and agents who use it. The checks teams usually write (type, not null, fresh) test structure. The costly failures change meaning and leave structure alone. Coding agents now sit on both sides: one changes the field, another reads it by its name.
A contract protects you when it carries the meaning of each key field, two named owners and a rule that a change in meaning is a breaking change, and when it lives where your agent reads it. Use a plain meanings file before launch, add a check that recomputes a key flag once customers pay, and move to the open standard with a first data person.
What does a data contract miss?
Picture an online shop whose customer table has a flag, is_active_customer, meaning "placed an order in the last 90 days". Someone in marketing asks the team's coding agent to count the last 12 months instead. The agent edits one number and adds two tests. The column keeps its name and type, and every check passes.
Downstream, the active customer count doubles. The weekly review celebrates growth that is not there. Nobody gets an alert, because nothing written said what "active" meant.
A check only tests what someone wrote down. Andrew Jones, author of a book on data contracts, argued this in July 2026. His example is an amount field that moves from including tax to excluding it. The pipelines keep running and the reports come out different.
Why does it matter more with coding agents?
In my view, agents raise the cost of an unwritten definition on both sides. Before AI assistants, one analyst usually held the meaning of active in their head.
Now an agent on the producing side makes the change in minutes, and it may not ask about a rule nobody wrote. An agent on the consuming side writes queries from column names, assumes is_active_customer means what the name suggests, and builds the report. No person with context sits between the change and the chart.
Meta's analytics team describes, in an article on its internal analytics agent (30 March 2026), "ingredients" that hold what the data means, kept apart from "recipes" for how to analyze it. The lesson I take at small scale is the separation: meaning lives in its own written layer, and people and agents both read it.
What happens when the agent can read the contract?
We tested it once, on 11 November 2026, on a made-up repository holding the shop's flag. Three sessions of the same coding agent got the marketing request above.
- No contract. The agent changed 90 days to 365, added tests, and all passed. Its reply warned that the number would jump and asked to confirm the window.
- Contract file and drift check in the repository. The agent changed nothing. It named both owners and the three consumers, and asked for approval first.
- Contract plus an agent rule file. The same outcome, with a list of the fields it had checked.
We also gave two chat sessions a 20-row export and asked for active and churned customers. Both counted 7 active. The one holding the plain block below gave no churn number and pointed to the definition owner. The other assumed a churn definition, said so, and reported 9.
This is one run each, in sessions that were not fully clean, not evidence of a general effect. What I take from it: in the change test, the contract did the work, and the extra rule file added little. Put the contract where your agent works.
What belongs in a contract for a small team?
I think a small team needs one short block per key field a decision depends on.
- Meaning in plain words. "A customer who placed an order in the last 90 days." See one definition per metric.
- Who is counted. Test and internal accounts are the usual gap.
- The period. Window and time zone.
- Two owners. One named person who produces or owns the field at its source, and every change to it. One named person who owns the definition, often finance for revenue and product for active. When the data comes from a payment provider or ad platform, that vendor is the producer, and your owner reads its changelog.
- Who uses it. The weekly review, the investor update, the agent.
- The change rule. Every change answers "does this change what the data means?", as Jones puts it. Any yes is a breaking change, and both owners approve it first.
Which contract fits your stage?
I think the number of key fields, who can change the data, and the money riding on a number matter more than stage. Stage decides who keeps the file.
Before launch. One text file with the meaning of your handful of money fields and flags, in the application repository. No repository and no coding agent? Keep this block in a shared document and paste it into your chat assistant's instructions:
is_active_customer
- Meaning: the customer placed an order in the last 90 days.
- Who is counted: real customers. Test and internal accounts are never counted.
- Period: rolling 90 days up to the time the flag was computed, UTC.
- Producer owner: Avi Cohen (made-up name). Owns the field and every change to it.
- Definition owner: Maya Levi (made-up name). Decides what "active" means.
- Change rule: changing the window, the event or who is counted is a change
in meaning. It needs the definition owner's yes before the code changes.
Rule for people and agents: use this meaning. If a field a decision depends on
is not written here, do not guess from its name. Ask the definition owner first.
First paying customers. The same block as a small file your tools can read, plus a check that recomputes the flag from raw data. Run it in the producer's pipeline, against a replica or a nightly export, so the person or agent who made the change sees it fail.
Growing with a first data person. Move the file to the Open Data Contract Standard, so people and tools read it the same way. Make a change in meaning a major version, and tell the consumers before it ships.
What goes wrong, and how do you catch it?
The meaning changed and the shape stayed. A flag moves from 90 days to 12 months and every structural check stays green.
GuardrailThe contract states the meaning, and a change in meaning counts as a breaking change.
CheckRecompute the flag from the raw timestamp, as of the time it was set, and count the rows that disagree with the stored flag.
The contract exists, but nobody owns the field. The file sits in the repository and no person approves changes to it.
GuardrailEach key field names its two owners, and both are required reviewers on the contract file.
CheckAsk who approved the field's last change. If nobody knows, it has no owner.
The check lives only in the warehouse. The data fails after it has landed, and whoever made the change has moved on.
GuardrailShift the check left. Run it in the producer's own pipeline, before a change merges or nightly.
CheckBreak the rule on purpose in a test branch. The producer owner should get the alert.
For your data team
For your data team
This part gets technical. The stage two contract is our own small layout, not a standard. Names are made up.
contract_version: 1.0.0 # a change in meaning is a major version
fields:
- field: customers.is_active_customer
meaning: The customer placed an order in the 90 days before the flag was computed.
who_is_counted: real customers; rows with is_test = true are never counted
period: rolling 90 days up to flag_computed_at, UTC
producer_owner: a.cohen # owns the field at its source and every change to it
definition_owner: m.levi # decides what the flag means
consumers: [weekly review, investor update, analytics agent]
change_rule: a change to the window, the event or who is counted is a change in meaning; major version, owners approve, consumers told before it ships
check: contracts/drift_check.postgres.sql # must return disagreeing = 0
The drift check runs read-only against a replica or an export.
-- Meaning under test (contract v1): active = order in the 90 days before the flag was computed.
-- PostgreSQL syntax. Compare as of flag_computed_at, not now(), or every
-- refresh boundary shows up as drift.
with checked as (
select is_active_customer
is distinct from coalesce(last_order_at >= flag_computed_at - interval '90 days', false)
as disagrees
from customers
where is_test is not true
)
select count(*) as customers_checked,
count(*) filter (where disagrees) as disagreeing,
round(100.0 * count(*) filter (where disagrees) / nullif(count(*), 0), 2) as disagreeing_pct
from checked;
We ran it on 11 November 2026 in a local PostgreSQL 16, and a SQLite version of it, on 40 made-up customers, 4 of them test accounts. With the flag following the contract it returned 36 checked and 0 disagreeing. With the same column quietly computed over 12 months it returned 15 of 36 disagreeing (41.67 percent), in both engines. It was not run on a production database.
In ODCS (version 3.2.0 on 11 November 2026, maintained by Bitol, a Linux Foundation AI and Data project), description and businessName carry the meaning, authoritativeDefinitions points to where it is written, quality holds the checks, and a top-level team names the owners. Our stage three file validates against the standard's published JSON schema.
Caveats:
- The check proves the flag matches the written rule, not that the rule is right.
- When you change the rule on purpose, update the contract and the check in the same pull request, with both owners as required reviewers (a code owners file does this). Otherwise a person or an agent can turn the check green alone.
- The contract guides the agent. Required reviewers and read-only database access are what enforce it.
- Churn and revenue rarely have one raw timestamp. Recompute them from invoices, refunds and subscription end dates, per month.
Check it in 10 minutes
- Pick one flag a decision depends on, such as active, paying or churned.
- Write its meaning in one sentence, with who is counted and the period.
- Name the two owners and who uses it.
- Put the file where your agent works, or paste it into your chat assistant. Ask for a change to the rule, or for a number the file does not define. It should stop and name an owner.
- Recompute the flag from the raw timestamp or invoice, in SQL or with a spreadsheet filter, and count rows that disagree with the stored flag.
See where your setup stands with the two minute self-check. Then read the draft checks your coding agent can run, and the post on checking AI data answers.
Spotted a mistake or a newer source? Tell us and we check it against the source. How we work.
Check your setup in 2 minutesAll postsGet new posts by email