The real work in a data agent is the harness around it

What is a harness?

Think of a car engine. On its own, on a stand in a workshop, it is powerful and useless. It becomes a car only once you add steering, brakes, a fuel line, gauges and a seat for the driver. Nobody buys an engine to drive to work.

An AI model is the engine. It reasons and writes queries fast. The harness is the rest of the car: the instructions it reads, the tools it may use, what it is allowed to touch, what it remembers between steps, the checks its work must pass, and the points where a person approves before it continues. A common way to put it, as Addy Osmani explains in his overview of harness engineering: agent equals model plus harness.

Where does the term come from?

Software has used "harness" for a long time, for the code that runs a program under test. The current sense comes from AI coding. There, the model writes the code, and engineers spend more and more of their effort on the harness around it: instructions split into small files instead of one giant prompt, context kept between sessions, checks such as tests that must pass, a clear definition of "done", and a gate where the agent stops and asks. People now call this harness engineering.

I think data work needs the same shift, and needs it more, because a wrong data answer looks exactly like a right one. A failing test is loud. A wrong number is quiet.

Why does a data agent need one?

Picture a small team that ran three ad campaigns last month and asks its AI assistant, "Which campaign brought us the most customers?" The assistant joins sign-ups to campaign tags and names a winner in one confident sentence.

Behind that sentence sit four choices nobody made on purpose. It counted sign-ups, not paying customers. It gave each sign-up to the last ad clicked, although the team usually credits the first. It included the sign-ups the team created while testing the landing page. And it read "last month" in a different time zone from the ad platform. Any one of these alone can change the winner. The team moves budget to it anyway, because the answer was fluent.

The model was not broken. Nothing around it said what "customer" means here, which attribution rule the team uses, what to leave out, or when to stop and ask. A better model alone does not close that gap.

What is a minimal harness for a data agent?

The agent tool already gives you the loop, the tools and a way to read files. What it cannot know is your business. Here are the five data parts you add, in the order I would build them.

A rule file. One short document the agent must follow on every task. It says which sources the agent may use and how it must answer.
Metric definition files. One small file per key number, covering what it means, who is counted, which period, and who owns it. The agent must cite the file behind every number. The one definition per metric post covers these in detail.
Read-only access to a copy of the data. Never the live database. The agent can look and calculate, but its credentials cannot change anything, and its queries run on the copy instead of the system your customers use. The no-analytics-on-primary check covers this.
A short list of checks before an answer counts as done. For example: test accounts excluded, period stated, definition cited. An answer that fails a check goes back to the agent. It does not reach the reader.
A stop rule. When no definition covers the question, the agent says "I don't know" or asks which definition you mean. It does not invent one.

Four parts are text files. The fifth is a copy of the data and a login that can only read it.

The agent drafts an answer

Checks before it counts as done

The agent reads the rule file and a metric file, then queries a read-only copy; the live database stays closed to it. Before an answer counts as done, it must pass every check. A draft that fails goes back to the agent with the failed item named, a question no metric file covers hits the stop rule and goes to a person, and only a draft that passes every check reaches the reader.

What does it look like at each stage?

I think the harness should grow in the same order as the company, and no faster.

Before launch. You have one or two questions and almost no data. Write the rule file, plus a one page glossary for the one or two numbers you need. Give the agent a copy of the data to read, even a spreadsheet export. Add one check: every answer names its definition.

First paying customers. Money now flows through billing, so wrong numbers cost more. Turn each glossary entry into its own metric file. Make the read-only copy a scheduled copy of your production database and billing data. Write the checklist down, with the three example checks as the minimum. One named person owns the files.

Growing with a first data person. Questions arrive faster than one person can review. The data person turns the checks into code that runs before an answer is shown, logs every answer with the files it cited, and reviews any change to a metric file. Questions the agent declines go to them.

What goes wrong, and how do you catch it?

The agent invents a definition. It answers with a meaning of "active" that exists nowhere in your company.
Guardrail: the stop rule, plus a rule that every number cites a metric file.
Check: ask a question your files do not cover. A good harness answers "I don't know" or asks you to choose a definition.

The agent claims it is done, but it is not. It reports a number without noting that test accounts were included.
Guardrail: the checklist, with a clear definition of done.
Check: open the log for the agent's last answer and look for each check's result. An agent saying it ran a check is not proof; a recorded result is. If there is no log, start one.

The agent touches live data. A badly formed query slows the product, or a write changes a real record.
Guardrail: read-only access to a copy, never the live database.
Check: ask your engineer to try a write with the agent's credentials, and to confirm those credentials cannot reach the live database. The write must be refused.

For your data team

This part gets technical. The harness is a folder in your repository, changed by reviewed pull request, and read by the agent at the start of each task. A starting rule file might look like this:

# agent-rules.yaml  (stage: first paying customers)
version: 2
sources:
  allowed: [analytics_replica, billing_export]    # read-only copies
  forbidden: [production_primary]
access:
  mode: read_only
  credentials: replica_reader                      # no write grants
metrics:
  folder: metrics/
  rule: >
    Compute a reported number only from a metric file.
    Cite the file name and version in the answer.
before_done:                                       # every item must pass
  - test_and_internal_accounts_excluded            # is_test, is_internal
  - period_stated                                  # dates and time zone
  - definition_cited                               # file name and version
stop_rule:
  when: no_metric_file_covers_the_question
  do: ask which definition to use, or answer "I don't know"
changes:
  metric_files: pull_request_with_named_reviewer

Notes on how the pieces fit:

Enforcement versus guidance. The rule file guides the agent. Permissions enforce it. Credentials with no write grants, scoped to the replica and the export only, hold even if the agent ignores a line of text. A read replica also keeps heavy queries off the primary, though very long queries can still delay replication on some setups, so set a statement timeout.
Checks as code. At first a person runs the checklist by hand. Later, write each check as a small test, for example a count of is_test = true rows behind the answer that must be zero, and run it before the answer is shown.
Logging. Store each question, the files cited, the checks passed and the answer, so you can replay how a challenged number was made. A question the agent keeps declining is a signal to write a new metric file.

Check it in 10 minutes

Ask your agent the question you ask most often, for example which campaign brought the most customers last month. Write down the answer.
Ask which definition it used and where that is written. If it cannot name a file, you have no harness for that number yet.
Ask a question your files do not cover. It should say "I don't know" or ask you to choose.
Look at the record of which checks passed for that answer: test accounts, period and definition. If nothing was recorded, nothing was checked.
Ask your engineer whether the agent can write to the live database. The answer should be no.

See where your setup stands with the two minute self-check, or read the draft checks your coding agent can run.