One supervising agent moves the single point of failure up a level. A refined system removes it.
This morning, Chandrakumar R Pillai, an AI advisor and enterprise architect based in Brussels, devoted his AI Daily Nutshell newsletter to a framework called Shepherd. [Link to Chandrakumar’s post]
According to his summary of IEEE Spectrum’s reporting, researchers at Northeastern University and Stanford University built Shepherd to manage AI agents that go off track. Their paper is due to be presented at NeurIPS in December. The idea borrows version control from software engineering. Every action a worker agent takes is saved as a snapshot, so a supervising “meta-agent” can watch the work, catch a faulty action, rewind the agent to an earlier step, or send it down a different path. Risky actions, such as sending an email, are held in a sandbox until a meta-agent or a human approves them.
Chandrakumar’s own reading of it is a good one. Most AI safety debates focus on preventing mistakes. Shepherd assumes mistakes will happen and makes them reversible. He connects that to the EU AI Act’s requirements for logging and human oversight, and he closes with five controls every agent deployment needs.
Then he lists the questions that remain. The first one is the right one: who watches the shepherd?
The problem one level up
A meta-agent can make mistakes too. If the shepherd shares a blind spot with the flock, a confident error does not get caught. It gets approved, and the approval makes it look safer than it was.
That is the part of the design that a version history cannot fix. A complete record of what an agent did tells you what to undo. It does not tell you whether the review that let the action through deserved your trust.
My response
Here is the comment I posted on his thread:
Chandrakumar R Pillai, your first open question is the right one: who watches the shepherd? A single meta-agent supervising the workers only moves the single point of failure up one level. If the shepherd shares a blind spot with the flock, the error passes through confidently and gets approved.
That’s why we don’t rely on one supervisor. Our ARA™ RAM 2025™ intelligence engine, running on SLAP OS™, puts the same question to multiple models from different makers at the same time, organized as a main panel and a Shadow Panel™ that reviews the work independently. The model that writes an expectation never scores it, and the human orchestrator makes the final call.
Shepherd’s version history records what an agent did. Our Litmus Matrix records how every reviewer reasoned about every question, round by round, and disagreement stays visible instead of being smoothed away. That reasoning record travels with the work, so the next reviewer sees not just the answer but how it was reached and where it was challenged.
So to your question, should agents report to an AI or to a human? Neither alone. The overseers should check each other, and a human should be accountable for the outcome.
What the reasoning record looks like
This is not a diagram of how the system should work. It is the record of how it did work, on a real post.
SLAP OS™ Multimodel Litmus Test™: one post, five rounds of review, 14 October 2026. Each cell records one reviewer’s engagement with one question: clears the bar, qualified, or must fix. Each round is confined to itself and is not comparable across rounds. Rust cells are expected; the instrument exists for progressive insight, not progressive approval.
Before It’s Too Late to Put the AI Genie Back in the Bottle was published on 14 October, it went through five rounds:
- The main panel reviewed the original draft.
- The Shadow Panel reviewed the same draft. Its only job was to find the weaknesses critics would attack.
- The Shadow Panel reviewed again, comparing the revised draft against another post without knowing which was which.
- The main panel reviewed the revision.
- Model 9 read it as a general-workforce reader would.
In Round 2, all three Shadow Panel reviewers named a defect on four of the six questions they assessed. That is what a reviewer whose only job is finding flaws should look like. If the rust cells had been missing, the panel would not have been doing its job.
Why the reasoning matters more than the record
Shepherd’s history answers the question what happened? The Litmus Matrix answers a different one: how much should we trust the decision to let it happen?
Three things make that possible, and none of them is a single smarter supervisor:
- Reviewers who can’t share the same blind spot. The models come from different makers, so a confident error from one shows up next to answers that don’t contain it. The disagreement tells the human orchestrator where to look.
- Separation between writing and scoring. The model that writes an expectation never scores it. Nobody grades their own work.
- Disagreement that stays on the record. Rust cells are not driven out to make a result look settled. A round where everyone agrees too early is treated as a warning, not a success.
The human still makes the final call. The difference is that the call is made with the full reasoning in view, including every point where a reviewer pushed back.
Chandrakumar is right that “undo” may be the most important feature in agentic AI. We wrote about what happens when the correction arrives from outside, after release, in No Loop, No Learning. An undo button helps after the fact. A reasoning record is how you need it less often.
The answer to his question
Should AI agents report to another AI, or directly to a human? Neither alone.
A shepherd watching the flock is an improvement. A shepherd that nobody watches is the same risk, one level up. The overseers have to check each other, from different starting points, with their disagreements left visible, and a named human has to be accountable for the outcome.
That is not a future design. It is how Hansen AI Labs™ has worked on every post, lab and assessment we have published, and the record is there to read: Why ARA™ RAM 2025™ SLAP OS™ Activates and Manages AI Grounding.
-30-
The labs behind the record
Hansen AI Labs™ tests the process in real time, on live events, with expectations locked before the outcome is known and scored afterward. These are the four labs on the record so far:
Across our labs, the output accuracy of ARA™ RAM 2025™ running on SLAP OS™, assessed against the evidence, is holding at an average of around 91 percent, and it fluctuates. That is a different measure from our prediction record, which includes misses as well as hits. Both are tracked, because the misses and what caused them teach us as much as the hits do.
The Isaias Strand Lab record is available under NDA to qualifying organizations. Contact us at HPT@hansenprocurement.com.
Related
Who Watches the (AI) Shepherd? The Answer Is Not Another AI Shepherd
Posted on November 20, 2026
0
One supervising agent moves the single point of failure up a level. A refined system removes it.
This morning, Chandrakumar R Pillai, an AI advisor and enterprise architect based in Brussels, devoted his AI Daily Nutshell newsletter to a framework called Shepherd. [Link to Chandrakumar’s post]
According to his summary of IEEE Spectrum’s reporting, researchers at Northeastern University and Stanford University built Shepherd to manage AI agents that go off track. Their paper is due to be presented at NeurIPS in December. The idea borrows version control from software engineering. Every action a worker agent takes is saved as a snapshot, so a supervising “meta-agent” can watch the work, catch a faulty action, rewind the agent to an earlier step, or send it down a different path. Risky actions, such as sending an email, are held in a sandbox until a meta-agent or a human approves them.
Chandrakumar’s own reading of it is a good one. Most AI safety debates focus on preventing mistakes. Shepherd assumes mistakes will happen and makes them reversible. He connects that to the EU AI Act’s requirements for logging and human oversight, and he closes with five controls every agent deployment needs.
Then he lists the questions that remain. The first one is the right one: who watches the shepherd?
The problem one level up
A meta-agent can make mistakes too. If the shepherd shares a blind spot with the flock, a confident error does not get caught. It gets approved, and the approval makes it look safer than it was.
That is the part of the design that a version history cannot fix. A complete record of what an agent did tells you what to undo. It does not tell you whether the review that let the action through deserved your trust.
My response
Here is the comment I posted on his thread:
What the reasoning record looks like
This is not a diagram of how the system should work. It is the record of how it did work, on a real post.
SLAP OS™ Multimodel Litmus Test™: one post, five rounds of review, 14 October 2026. Each cell records one reviewer’s engagement with one question: clears the bar, qualified, or must fix. Each round is confined to itself and is not comparable across rounds. Rust cells are expected; the instrument exists for progressive insight, not progressive approval.
Before It’s Too Late to Put the AI Genie Back in the Bottle was published on 14 October, it went through five rounds:
In Round 2, all three Shadow Panel reviewers named a defect on four of the six questions they assessed. That is what a reviewer whose only job is finding flaws should look like. If the rust cells had been missing, the panel would not have been doing its job.
Why the reasoning matters more than the record
Shepherd’s history answers the question what happened? The Litmus Matrix answers a different one: how much should we trust the decision to let it happen?
Three things make that possible, and none of them is a single smarter supervisor:
The human still makes the final call. The difference is that the call is made with the full reasoning in view, including every point where a reviewer pushed back.
Chandrakumar is right that “undo” may be the most important feature in agentic AI. We wrote about what happens when the correction arrives from outside, after release, in No Loop, No Learning. An undo button helps after the fact. A reasoning record is how you need it less often.
The answer to his question
Should AI agents report to another AI, or directly to a human? Neither alone.
A shepherd watching the flock is an improvement. A shepherd that nobody watches is the same risk, one level up. The overseers have to check each other, from different starting points, with their disagreements left visible, and a named human has to be accountable for the outcome.
That is not a future design. It is how Hansen AI Labs™ has worked on every post, lab and assessment we have published, and the record is there to read: Why ARA™ RAM 2025™ SLAP OS™ Activates and Manages AI Grounding.
-30-
The labs behind the record
Hansen AI Labs™ tests the process in real time, on live events, with expectations locked before the outcome is known and scored afterward. These are the four labs on the record so far:
Across our labs, the output accuracy of ARA™ RAM 2025™ running on SLAP OS™, assessed against the evidence, is holding at an average of around 91 percent, and it fluctuates. That is a different measure from our prediction record, which includes misses as well as hits. Both are tracked, because the misses and what caused them teach us as much as the hits do.
The Isaias Strand Lab record is available under NDA to qualifying organizations. Contact us at HPT@hansenprocurement.com.
Share this:
Like this:
Related