A studio builds an agentic decision system and lives in it for a year. Contracts, gates, verification captures, escalation thresholds. It works, and the work is real: cycle times down, regressions caught, evidence flowing.
Then the system ships to a partner organization. Same agents. Same contracts. Within a month, the partner is either drowning in gates that stop work nobody considers risky, or sailing through gates that catch nothing anyone considers dangerous.
The agents did not get worse. The judgment they enforce was trained somewhere else.
What judgment is made of
Judgment sounds like a personal gift, but operationally it is a local product with four ingredients:
- Pattern memory: what happened here, the last several times something looked like this.
- Consequence memory: what being wrong actually cost, and who paid it.
- Risk appetite: how much damage the group can absorb, which is a budget question before it is a personality question.
- Timing: how fast this particular ground moves, and therefore how long a decision can stay open.
All four ingredients are trained on local conditions. This is why judgment is the scarce product in review and curatorial work, and why two competent people can look at the same diff and honestly disagree. They were not trained on the same cases.
Where agents inherit it
An agentic decision system is judgment made executable. Every contract field encodes somebody's answer to a local question:
| System component | The judgment it encodes | Trained where |
|---|---|---|
| Acceptance criteria | What counts as good enough here | This group's history of shipped work |
| Verification thresholds | How much proof a claim needs | What being wrong cost this group |
| Escalation triggers | When to stop and ask a human | Which failures this group fears |
| Definition of done | What the group will not say yes to | Every miss the group still remembers |
| Retry and rollback limits | How much damage is recoverable | The group's actual recovery capacity |
The table is the point of this post. None of those rows is universal. All of them feel universal to the group that wrote them, because from inside a village its assumptions look like reality.
Thresholds are cultural
Consider a verification threshold: a pull request needs two passing test suites and a recorded browser check before done.
In one organization that is a floor, built after a painful year of polite lies shipped to production. In another it is a ceiling, imposed on a team whose actual risk is shipping too slowly to survive the quarter. Same threshold, opposite judgments: one group reads it as the minimum honesty requires, the other as the maximum survival allows. The number does not carry its own meaning. The ground writes the meaning, and the meaning is invisible until the threshold crosses a border.
This is the sharp version of the series argument. A decision system is not neutral machinery that judgment gets applied to. A decision system is judgment, crystallized. When the system moves, the judgment moves with it, wearing the costume of objectivity. Thresholds are the most confident parts of the system, and they are exactly as local as everything else.
Recalibrate on site
Treat a shipped decision system like a transplant, because it is one. The receiving ground needs a recalibration period, and it needs to be explicit:
- Sit with the receiving group and rewrite the definition-of-done examples in their own cases, not yours. The skeleton stays; the flesh is local.
- Collect their failure stories, the misses they still remember, and let those stories set the escalation triggers. Their fears are data.
- Run the gates in shadow for a cycle: the system observes and records what it would have blocked, nobody is blocked yet, and the group argues with the log. The arguments are the calibration.
- Expect the thresholds to move, and let them. If nothing moved during recalibration, nobody was honest.
What survives the transfer is the skeleton: claims need evidence, evidence needs capture, thin evidence escalates, and done is a contract rather than a mood. Those principles are ground-independent. Every specific value that implements them is not.
The uncomfortable summary
The promise of agentic systems is consistency: the same rules, applied the same way, every time. That promise is real, and it is exactly the risk. Human judgment drifts with conditions, which is sometimes its best feature. An agent system does not drift. It enforces last year's ground on this year's, or this ground's assumptions on that one's, indefinitely and at scale.
So the question to ask before any deployment stops being "does this system work?" It was never answerable anyway. The question is "whose judgment is this, trained on which conditions, and how far is the receiving ground from home?"
FAQ
Are you saying AI systems cannot be generalized?
The models generalize; the thresholds do not. A model trained broadly can perform everywhere. The contracts that govern what counts as done, when to escalate, and how much proof is enough are organizational artifacts, and they transfer about as well as any management practice does, which is to say: only with recalibration by the people who will live under them.
Does more data fix this?
It depends which data. More history from the home group deepens the home bias. Data from the receiving ground, especially its own misses and its own near-misses, is what recalibration actually consumes. The fix is not volume. It is provenance.
How long should recalibration take?
One honest cycle is enough to know if it will work, because the shadow-mode arguments surface every mismatch quickly. Expect the full pass to take a few cycles of real work. If the receiving group has no failure stories to calibrate against, slow down: a group that has never survived a miss has no local training data, and the project needs that before it needs software.
Is there a shortcut for small teams?
Yes, one. Send the system's authors to sit with the receiving team through the first real decisions, rather than sending documentation. Judgment transfers between people who watch the same work and argue about it. It does not transfer through configuration files.
Related reading
- Confident Wrong Output Marked Done Is the Agentic Failure Mode
- Agent Conversations Without Contracts Are Faster Confusion
- Judgment Is the Scarce Product in Code Review and Curatorial Work
- Human Judgment Is the Scarce Layer
- AI Needs a Harness Before It Needs Autonomy
- A Decision System Only Works on the Ground That Grew It
- Ten Years Is Not Ten Years of Experience
If you are shipping decision systems across organizations, book a discovery call.