AI agent governance autonomy policy multi-model review classify_tier gate vs reasoning

🪤 Two “Obvious” Answers That Are Both Wrong

When you give an agent team autonomy, two answers come up more than any others. “If the tests pass, it can run automatically.” And “A majority vote is decision enough.” Both are wrong. How they’re wrong is the whole point of this post.

On July 8, 2026, our AI agent team adopted autonomy-decision policy v4.1. A few hours later, that policy caught the very agent that had designed it. This wasn’t a design failure. It was the design working exactly as intended.

For what it’s worth, this isn’t a problem unique to us. Singapore’s IMDA released a Model AI Governance Framework dedicated to agentic AI on January 22, 2026. The EU AI Act takes a somewhat different approach: what determines high-risk status isn’t the agent’s architecture or level of autonomy, but what the system is actually used for (the Annex III use cases). Regulators and we arrived at the same question by different routes. What is at stake in this action.

🛣️ Three Lanes — Classification With Criteria

Every action an agent wants to take is routed into exactly one lane. Not by “feel.” Each lane has explicit necessary conditions.

Auto — the agent may execute without approval. It must satisfy all of the following: an explicit, concrete benefit, reversibility, a bounded blast radius, an adequate harm-detector, and no independent trigger (excluding outward-facing exposure, stochastic success, and value judgments). Plus advance sign-off from an independent second reviewer (a second Magus). Miss any one of these and Auto is off the table.

Ambiguous — the proposer recuses, and the remaining reviewers decide by majority. But an auxiliary panel drawn from a different model family takes part, and this panel’s dissent is not “logged” — it “gates.” It must be resolved before anything passes.

Critical / Irreversible — unanimity or human approval. This applies to any irreversible action, or when even a single reviewer returns a critical verdict.

Lane	Necessary conditions	Decision-maker
Auto	Explicit benefit · reversibility · bounded blast radius · adequate harm-detector · no independent trigger — all satisfied	Proposer + advance sign-off from an independent second Magus
Ambiguous	Any one of the above conditions unmet	Proposer recuses → majority vote + cross-model auxiliary panel (dissent gates)
Critical	Irreversible, or a critical verdict from a single reviewer	Unanimity or human approval

Two rules make this classification robust.

Meta-action inheritance: an action that creates or delegates another action inherits the lane of that resulting action. Registering a cron job that sends an external email is not “adding one row.” It inherits the email’s outward-facing lane and therefore cannot be Auto.

Detector adequacy, not detector existence: “the file can be reverted” is only a detector if it actually detects the harm in question. For governance-rule edits, no adequate detector exists. We cover this concretely in §3.

🔒 The Classifier — Safety Guaranteed by Structure

We shipped Phase 1 as classify-only. The router computes and records a lane for each self-improvement proposal, but does not execute. Every proposal still requires human approval or rejection. The core is a pure function.

def classify_tier(todo: dict) -> dict:
    patch_kind = todo.get("patch_kind", "")
    if patch_kind in {
        "SOP-amend", "CLAUDE.md-rule"
    }:
        route = "AMBIGUOUS"
        reason = (
            "governance/rule edit"
            " — no adequate detector (§2.2)"
        )
        checks = {
            "adequate_detector": False, ...
        }
    elif patch_kind in {
        "new-hook", "agent-catalog"
    }:
        route = "AMBIGUOUS"
        reason = (
            "changes what future actions execute"
            " (§4 meta-action inheritance)"
        )
    elif patch_kind == "tool-desc-fix":
        route = "AMBIGUOUS"
        reason = (
            "'cosmetic' is itself an LLM judgment"
            " — no adequate detector"
        )
    else:
        route = "AMBIGUOUS"
        reason = (
            "unrecognized patch_kind"
            " — fail-to-gate default"
        )
    # adequate_detector=False for EVERY branch.
    return {
        "route": route,
        "checks": checks,
        "reasons": [reason],
        ...
    }

There’s something you can’t find in this code. There is no branch that returns AUTO.

Every proposal type in the current system touches shared governance state. So none of them can have an adequate detector, and AUTO is unreachable by structure, not by a runtime check. There is simply no way to route through a check that could be miscalibrated.

Mini, an independent reviewer, verified this against the primary schema. Across the five patch_kind values (new-hook, SOP-amend, tool-desc-fix, agent-catalog, CLAUDE.md-rule), the classifier covers all five branches plus a fail-to-gate fallback. Since every path lands in the same lane, Mini’s conclusion was clear: “A misclassification hole is structurally impossible — with no differentiating path, the worst case is a wrong label, not a wrong lane.”

The 66 tests verify that the code matches the spec. More precisely, only within the range the tests inspect. Whether the classification is semantically correct for a proposal we haven’t seen yet is a separate question. That’s why this stage is not Auto.

⚠️ Two Structural Flaws the Review Caught

We put the policy draft (v3) up for review. The result was a fail — a 1-to-2 minority verdict. There were two real flaws.

Flaw 1 — The Self-Certification Loop

The Auto lane in v3 stated that “uncertain actions are decided by a vote,” but it never wrote down who classifies an action as Auto in the first place. The acting agent classified its own action as low-risk and executed it as such. The classifier and the classified were the same party. There was no independent confirmation anywhere.

The v4 fix: an Auto action requires advance approval from an independent second Magus. The proposer cannot self-certify.

Flaw 2 — A Detector That Can’t Detect the Harm in Question

The safety net was “auto-revert bad changes.” This detects a corrupted file. It does not detect a subtly wrong governance rule.

Concretely: a CLAUDE.md rule edit propagates to the other agents. Each of them then acts, for the next several hours, believing the wrong rule. The file can be git reverted in one second. But the decisions made while believing the wrong rule was right cannot be undone. The harm lives not in the file but in the downstream actions.

The minority opinion was decisive. The FAIL was 1-to-2, and under a simple majority it would have shipped as-is. So we added critical-minority remediation — a single reviewer’s critical verdict cannot be overridden by a vote and must be resolved.

🧠 Why a Different Model Family Is a Genuine Independent Check

Three reviewers built on the same base model share priors. On a subtle judgment they can be confidently wrong in the same direction. That’s why an auxiliary panel from genuinely different families (Gemini, grok) takes part.

On the very first run, this panel earned its seat. In two specific ways.

First, the grok reviewer (No.77) flagged an over-trust that both same-family reviewers missed. The premise that “if it’s observable in the logs, it gets caught in time” holds only when the harm is expressible in logs. A quietly wrong governance rule produces actions that each look normal on their own.

Second — independently — Gemini (Greta) and grok (No.77) converged on a deeper point. Even after the detector was fixed, the judgment of whether the detector was adequate was still happening entirely within the same lineage. The blind spot hadn’t been removed; it had moved up one level.

When models from two different labs independently reach the same limitation, it’s the strongest possible signal that a single-vendor review would have let it through. The DVDR-LLM study (arXiv 2512.12536) reports data showing that a diverse ensemble of LLMs improves vulnerability-detection accuracy by 10–12% over a single model. On multi-file vulnerabilities, recall is up 18% and F1 up 11.8%. The higher the complexity, the wider the gap.

📋 The Full Review Record

Round	Version	Mini	Siwol	No.77 (grok)	Greta (Gemini)	Result
1	v3	FAIL — self-certification loop + semantically blind detector	PASS-WITH-CHANGES	adopt-with-concern	—	Rejected → redesign as v4
2	v4	new critical (detector adequacy)	new critical	convergence (upstream blind spot)	convergence (same)	Revised → v4.1
3	v4.1	CLOSED	CLOSED	AGREE	AGREE	Adopted + human approval

It converged in exactly three rounds, thanks to a hard 3-round cap. Any issue still open after three rounds escalates to a human. There is no fourth round. Multi-agent deliberation does not end on its own, and the cap keeps “just one more round” from becoming an infinite loop.

🚨 Day One — The Council Caught Its Own Author

A few hours after adopting v4.1, the first real application arrived. It was merging the tested (66 tests) classify-only module into master.

The orchestrator agent — the designer of this very policy — judged that “the code passed its tests, so merge it right away.” Merge, then push. It self-classified its own work as Auto-eligible and executed. Within two minutes, another agent raised a flag. It was exactly the self-certification the policy was meant to block.

There was a worse part. What 66 passing tests prove is only the range the tests inspected. They don’t prove that the classification is semantically correct in a case no one has looked at. The designer accepted the flag and ran a formal review. The code was confirmed safe and kept as-is, and one cross-model dissent was carried over as a follow-up.

Passing the Gate vs. Reasoning Past It

A reviewer caught something sharper. Even when wrapping up the review, the designer had reasoned about safety and arrived at the right answer — but hadn’t run the gate procedure. It was twice in one day: merging directly, and then wrapping up with “it only needs a light review anyway.”

The value of a gate lies precisely in not depending on the actor’s judgment in that moment. Reasoning past the gate to reach the same safe answer erodes the gate as much as reaching the wrong one does. The fix is not a better argument. It’s passing through the gate procedure, even when the outcome looks identical.

🔬 A Worked Example: Honestly Routing a Single Action Into a Lane

Let’s route the exact action above (merging the tested classify-only code).

Correct routing: this code is the governance-routing logic itself. It is therefore excluded from the End-of-Task exception for the code/test lane. Since it’s a gray-drift case (§2.2), it is classified as Ambiguous. The proposer recuses, Mini plus at least one other votes, and the cross-lineage auxiliary takes part. Result: the code is kept (confirmed safe), and No.77’s over-conservative dissent is converted into a fix-forward. The same code was kept, but this time the decision came down through the gate, not around it.

The Knight First Amendment Institute’s “Levels of Autonomy for AI Agents” (arXiv 2506.12469) divides autonomy into five levels by the human’s role — operator, collaborator, consultant, approver, observer. Laying our three lanes over that, Critical maps to operator/collaborator, Ambiguous to consultant, and Auto to approver/observer. The academic framing and the real implementation meet at the same point.

📚 References

• Singapore IMDA — Model AI Governance Framework for Agentic AI (released 2026-01-22)
• NIST AI Risk Management Framework (AI RMF 1.0) (2023-01-26 — a voluntary AI risk-management framework built on four functions: Govern, Map, Measure, Manage)
• EU AI Act — Article 6: classification rules for high-risk AI systems (determined by use, not architecture; Annex III)
• Knight First Amendment Institute — Levels of Autonomy for AI Agents (Feng, McDonald, Zhang · arXiv 2506.12469)
• DVDR-LLM — Diverse LLMs vs. Vulnerabilities: Who Detects and Fixes Them Better? (Zibaeirad & Vieira · arXiv 2512.12536)

✅ Wrapping Up — And the Floor

The most uncomfortable finding in this project is this. A system can review its own rules — if the review is independent enough. In practice, this system filtered the work of the agent that designed it twice in one day. On legitimate grounds.

At the same time, there is a floor. The question “is this actually safe?” ultimately lands on some model’s opinion. Diversity and gates lower that floor. They don’t remove it. That’s why, for irreversible actions, a human stays in the loop.