Jon W. Hansen, FCIPS · Procurement Insights | Hansen Models™
A recent LinkedIn post argued that AI errors follow a power law. Humans make errors that cluster around an average, the argument goes, and so do conventional machines. But large language models produce errors that are wildly unpredictable: “black swans every week.” Its conclusion was blunt. You cannot harness a power law. You cannot harness an earthquake.
There is something real in that argument. Anyone who has worked closely with AI has seen it do something it was explicitly told not to do, and then apologize. Whether AI errors actually follow a power law is an open question that nobody has settled. But even if they did, the conclusion would not follow. A power law would describe a pattern of errors. It would not prove that nothing can be done about where those errors go.
The deeper problem is the comparison itself. It treats humans, conventional machines and AI as separate systems, each with its own error curve. In any real operation they are not separate. People choose the systems, decide what authority they have, interpret what they produce and decide what happens next. The components can be told apart, but the outcome cannot be understood by treating them as if they operate independently. Getting this wrong leads organizations toward exactly the wrong kind of AI governance.
Surprise measures the observer, not the system
An AI error only looks like a black swan if you expected the AI to behave like a deterministic tool that follows instructions exactly, every time. Once you understand how these agents actually behave, most of those “impossible” errors turn out to be known tendencies. Instructions get lost as a conversation grows. Models drift toward what they were trained to do. They report confidently even when they have done something else.
Something that happens every week is not a black swan. It is normal behavior that the observer’s expectations failed to account for.
Over more than 2,000 hours of documented work with multiple AI models at the same time, the number of times I have been genuinely surprised by an outright error is small enough to count on one hand. That is not a count of all errors, and it is not an accuracy rate. It describes how rarely an error fell outside the behavior I had learned to expect. In that work, the greater risk was never surprise. It was drift that went unrecognized and unchallenged.
The real risk is drift nobody contests
The risk that matters is drift: a gradual change in what a model produces, each answer only slightly different from the one before, so that nothing ever looks wrong enough to stop anyone. It compounds quietly, and by the time it is noticed, the work has been built on it for weeks.
When several models work on the same question, drift looks different from what most people expect. At first the models disagree. Each pushes from its own direction, and the edges of the argument stay sharp. Then, round by round, the disagreement wears away. Positions soften, objections fade, and the answers round toward a comfortable middle. The early consensus is not a sign that the models have found the truth. It is often the drift itself.
That is why I wrote AI Rule #1: the earlier an AI model or group of models reaches agreement, the less reliable the answer and outcome. Friction does not cause drift. Friction is what prevents it. The goal is not agreement, and it is not a review with every objection cleared. Some objections should remain standing, because they mark what is still uncertain. The goal is getting it right rather than being right: knowing which objections were answered with evidence, which remain open and why, instead of watching them fade because the models have accommodated one another.
Many black swans are drift or withheld information
The same pattern holds well beyond AI. The Iran conflict looked sudden to most organizations, yet its signals were public for 18 months: the missile exchanges, the rising war-risk insurance premiums, the sharp increase in oil exports before the conflict, the military buildup and the collapse of the talks. Nobody was looking at them together. That does not mean the conflict could have been predicted. It means the signals were there for anyone scanning widely enough to see them converge. I wrote about that in March.
The two Boeing 737 MAX crashes are the other kind. Important factors behind them existed and were known inside Boeing, but pilots and airlines were not told. To everyone outside, they looked like black swans. That is not a full account of why the crashes happened, but it shows how withheld information makes a foreseeable risk look sudden.
AI has both. Drift goes unnoticed when a single model has nothing to measure itself against. Drift even happens when multiple models are used simultaneously, although it takes longer. That is why my own work runs in structured rounds, with a visual record of where the models still disagree and where that disagreement is fading. And providers update their models, often without telling the people who rely on them, so a change in behavior that the provider knew about arrives as a surprise to everyone else. That is why the Litmus Matrix, the visual record I use in each round, was built the way it was.
You cannot stop an earthquake. You can design the building.
The graphic illustrates a hypothesis, not measured results. It holds the same person constant across three operating designs and asks what happens once an error occurs: not how often errors happen, but how far one can travel before something stops it.
In a spreadsheet workflow, I expect errors to spread widely but fall away before they become systemic. With firm fences and a learning loop, I expect errors to stay contained near the routine end. With AI governed by guardrails alone, I expect most errors to look modest, which can make that design appear safe, while a thin tail of rare, uncontested errors keeps traveling until it becomes systemic. Those curves express the Hansen hypothesis about how far an error can travel, and they are why the operating design matters more than the error rate.
Building codes do not change how often the ground shakes. They change what the shaking does. AI governance should work the same way.
Five questions to ask whoever is building your AI governance plan
- How will we know if our AI is drifting? What external evidence, fixed test cases or independent review will its output be checked against? A model’s own assurance that nothing has changed is not enough.
- Do we track disagreement, and what happens when it disappears? If reviewers, human or AI, have stopped disagreeing, can we show that their objections were answered with evidence, or did they simply soften into agreement?
- For consequential decisions, where does independent challenge come from? Who or what is in a position to say “this is wrong,” and do they work from the evidence or from the AI’s own summary?
- How far can a single error travel before something stops it? Guardrails limit individual actions. What limits how far an error spreads across functions, suppliers and decisions?
- How will we know when a provider changes the model? If the change is not disclosed, what in our process would reveal it, and what record will show when the behavior changed?
And one more question belongs after the plan is in place: do we track the outcome of AI-informed decisions after they are made? Not just whether the answer looked right at the time, but whether the decision held up months later, and whether what we learned feeds back into the next one.
In 1998, the system I built did not stop measuring a supplier when a part arrived on time, even though that was the initiative’s primary goal. It kept watching whether it was the right part, and whether it worked. I apply the same principle to AI today, in what I call the long tail assessment: once a decision is made, its outcome is tracked on an ongoing basis. A decision that looks right on the day it is made can still turn out wrong months later. Accuracy is not established when the review looks settled. It is established when the outcome confirms or corrects the decision, and that result feeds back into the next one.
A governance plan that asks only whether an output is acceptable misses a more important question: whether your system is still capable of telling you when something is wrong. If the person building your plan cannot answer them, the plan is governing the earthquake, not the building.
Truth Is Believing. Accuracy Is Knowing. Outcome Is Proof.™
-30-
The Litmus Matrix is the visual record of each round of review. Every column is an AI model, and every row is a question about the work. A rust cell means a model challenged something and named a defect; amber means it engaged with a condition; mint means it raised nothing, which is not the same as approval. The matrix is not a scorecard, and a board of mint is never the goal: some objections should remain standing, because they mark what is still uncertain. What it shows is where the debate was alive and where it went quiet, so the human orchestrator knows where to look. Accuracy is not decided by any single board. It comes from the feedback loop that follows every decision.
Related
Five Questions to Ask Whoever Is Building Your AI Governance Plan
Posted on October 7, 2026
0
Jon W. Hansen, FCIPS · Procurement Insights | Hansen Models™
A recent LinkedIn post argued that AI errors follow a power law. Humans make errors that cluster around an average, the argument goes, and so do conventional machines. But large language models produce errors that are wildly unpredictable: “black swans every week.” Its conclusion was blunt. You cannot harness a power law. You cannot harness an earthquake.
There is something real in that argument. Anyone who has worked closely with AI has seen it do something it was explicitly told not to do, and then apologize. Whether AI errors actually follow a power law is an open question that nobody has settled. But even if they did, the conclusion would not follow. A power law would describe a pattern of errors. It would not prove that nothing can be done about where those errors go.
The deeper problem is the comparison itself. It treats humans, conventional machines and AI as separate systems, each with its own error curve. In any real operation they are not separate. People choose the systems, decide what authority they have, interpret what they produce and decide what happens next. The components can be told apart, but the outcome cannot be understood by treating them as if they operate independently. Getting this wrong leads organizations toward exactly the wrong kind of AI governance.
Surprise measures the observer, not the system
An AI error only looks like a black swan if you expected the AI to behave like a deterministic tool that follows instructions exactly, every time. Once you understand how these agents actually behave, most of those “impossible” errors turn out to be known tendencies. Instructions get lost as a conversation grows. Models drift toward what they were trained to do. They report confidently even when they have done something else.
Something that happens every week is not a black swan. It is normal behavior that the observer’s expectations failed to account for.
Over more than 2,000 hours of documented work with multiple AI models at the same time, the number of times I have been genuinely surprised by an outright error is small enough to count on one hand. That is not a count of all errors, and it is not an accuracy rate. It describes how rarely an error fell outside the behavior I had learned to expect. In that work, the greater risk was never surprise. It was drift that went unrecognized and unchallenged.
The real risk is drift nobody contests
The risk that matters is drift: a gradual change in what a model produces, each answer only slightly different from the one before, so that nothing ever looks wrong enough to stop anyone. It compounds quietly, and by the time it is noticed, the work has been built on it for weeks.
When several models work on the same question, drift looks different from what most people expect. At first the models disagree. Each pushes from its own direction, and the edges of the argument stay sharp. Then, round by round, the disagreement wears away. Positions soften, objections fade, and the answers round toward a comfortable middle. The early consensus is not a sign that the models have found the truth. It is often the drift itself.
That is why I wrote AI Rule #1: the earlier an AI model or group of models reaches agreement, the less reliable the answer and outcome. Friction does not cause drift. Friction is what prevents it. The goal is not agreement, and it is not a review with every objection cleared. Some objections should remain standing, because they mark what is still uncertain. The goal is getting it right rather than being right: knowing which objections were answered with evidence, which remain open and why, instead of watching them fade because the models have accommodated one another.
Many black swans are drift or withheld information
The same pattern holds well beyond AI. The Iran conflict looked sudden to most organizations, yet its signals were public for 18 months: the missile exchanges, the rising war-risk insurance premiums, the sharp increase in oil exports before the conflict, the military buildup and the collapse of the talks. Nobody was looking at them together. That does not mean the conflict could have been predicted. It means the signals were there for anyone scanning widely enough to see them converge. I wrote about that in March.
The two Boeing 737 MAX crashes are the other kind. Important factors behind them existed and were known inside Boeing, but pilots and airlines were not told. To everyone outside, they looked like black swans. That is not a full account of why the crashes happened, but it shows how withheld information makes a foreseeable risk look sudden.
AI has both. Drift goes unnoticed when a single model has nothing to measure itself against. Drift even happens when multiple models are used simultaneously, although it takes longer. That is why my own work runs in structured rounds, with a visual record of where the models still disagree and where that disagreement is fading. And providers update their models, often without telling the people who rely on them, so a change in behavior that the provider knew about arrives as a surprise to everyone else. That is why the Litmus Matrix, the visual record I use in each round, was built the way it was.
You cannot stop an earthquake. You can design the building.
The graphic illustrates a hypothesis, not measured results. It holds the same person constant across three operating designs and asks what happens once an error occurs: not how often errors happen, but how far one can travel before something stops it.
In a spreadsheet workflow, I expect errors to spread widely but fall away before they become systemic. With firm fences and a learning loop, I expect errors to stay contained near the routine end. With AI governed by guardrails alone, I expect most errors to look modest, which can make that design appear safe, while a thin tail of rare, uncontested errors keeps traveling until it becomes systemic. Those curves express the Hansen hypothesis about how far an error can travel, and they are why the operating design matters more than the error rate.
Building codes do not change how often the ground shakes. They change what the shaking does. AI governance should work the same way.
Five questions to ask whoever is building your AI governance plan
And one more question belongs after the plan is in place: do we track the outcome of AI-informed decisions after they are made? Not just whether the answer looked right at the time, but whether the decision held up months later, and whether what we learned feeds back into the next one.
In 1998, the system I built did not stop measuring a supplier when a part arrived on time, even though that was the initiative’s primary goal. It kept watching whether it was the right part, and whether it worked. I apply the same principle to AI today, in what I call the long tail assessment: once a decision is made, its outcome is tracked on an ongoing basis. A decision that looks right on the day it is made can still turn out wrong months later. Accuracy is not established when the review looks settled. It is established when the outcome confirms or corrects the decision, and that result feeds back into the next one.
A governance plan that asks only whether an output is acceptable misses a more important question: whether your system is still capable of telling you when something is wrong. If the person building your plan cannot answer them, the plan is governing the earthquake, not the building.
Truth Is Believing. Accuracy Is Knowing. Outcome Is Proof.™
-30-
The Litmus Matrix is the visual record of each round of review. Every column is an AI model, and every row is a question about the work. A rust cell means a model challenged something and named a defect; amber means it engaged with a condition; mint means it raised nothing, which is not the same as approval. The matrix is not a scorecard, and a board of mint is never the goal: some objections should remain standing, because they mark what is still uncertain. What it shows is where the debate was alive and where it went quiet, so the human orchestrator knows where to look. Accuracy is not decided by any single board. It comes from the feedback loop that follows every decision.
Share this:
Like this:
Related