Your Agents Are Not Failing. Your Mandates Are
Why accuracy, audits, and committees cannot govern systems that act
The binding constraint
The dominant narrative says AI adoption is blocked by trust. Trust in outputs. Trust in vendors. Trust in organisations to deploy the tools responsibly. That narrative is comforting because it treats the problem as gradual and psychological. Better communication, more transparency, or improved model performance might erode it over time. It is also wrong in the way that matters most: it misidentifies the binding constraint entirely.
The binding constraint is legitimacy. Not whether a model produces impressive outputs. Not whether a dashboard looks reassuring. Legitimacy is whether a system is permitted to commit actions, under bounded authority, with enforceable constraints and real redress when it fails. Trust is what people say when they do not have enforcement. Legitimacy is an institutional property and an enforceable permission. Confuse them and you get governance theater: rituals that satisfy the need to appear governed without actually governing anything.
This distinction becomes unavoidable the moment systems move from suggestion to commitment. A copilot drafts; an agent executes. Drafting can be wrong without being catastrophic, because a human still decides whether to act on the output. Execution can be correct and still be illegitimate. A correct action taken without authority is not a success. It is an unauthorised exercise of power, and the correctness of the action does nothing to change that.
Most organisations cannot state authority in a form their systems can enforce. They describe it socially: roles, norms, escalation chains, policy documents that assume a human reader who will interpret context, negotiate exceptions, and absorb ambiguity. Automated systems do not absorb ambiguity. They operationalise it. When authority is implicit, the system exercises discretion that nobody bounded, and the organisation discovers this only after the discretion has produced consequences.
That is the adoption ceiling. Not a model ceiling. An authority ceiling. Verification costs stay high even as generation gets cheap because verification is doing work that authority should have done upstream. Organisations compensate for illegible mandates by scrutinising outcomes, adding approvals, and expanding review loops. None of that constrains runtime behaviour. It shifts liability onto people who lack control handles, and it will continue to do so until the underlying authority problem is addressed.
The power transfer is easy to miss because it arrives disguised as productivity. Platforms ship systems that can act. Operators deploy them because the upside is visible and the downside is diffuse. The risk does not disappear. It migrates to loss-bearers who cannot meaningfully contest decisions: frontline staff, end users, and institutions that inherit liability without corresponding control. If this framing persists, meaning organisations keep treating legitimacy as a trust problem, the migration will accelerate, and the loss-bearers will have no institutional mechanism to stop it.
This essay makes a narrow claim: an AI system is illegitimate if it can commit actions without a machine-checkable, revocable mandate. Technical quality does not rescue institutional illegitimacy. Accuracy does not grant permission. A well-behaved system can still be a power grab if its discretion is unbounded and its decisions cannot be contested. The question is not whether AI can be trusted. The question is whether authority can be made explicit, enforceable, and reversible under automation. The deeper question is who gains discretion when it is not.
Why this problem emerges now
The problem of delegating authority to systems that act on behalf of humans is not new. What is new is the speed, scale, and opacity at which the delegation occurs, and the degree to which existing governance infrastructure was never designed to handle it. To understand why agentic AI creates a legitimacy crisis rather than merely a technical challenge, it helps to look at how authority delegation has worked and failed in prior domains.
Financial systems spent decades developing infrastructure for delegated authority. A broker acts on behalf of a client. The scope of that authority is defined in a contract, bounded by regulation, and subject to fiduciary duty. When the broker exceeds that authority, there is a legal mechanism to contest it, a regulator to investigate, and a liability framework to assign harm. This infrastructure did not emerge voluntarily. It emerged because the absence of it produced catastrophic failures: rogue traders, unauthorised positions, and systemic collapses. It emerged also because the losses were large enough and visible enough to force institutional response. The governance apparatus of financial markets is, fundamentally, a response to the failure modes of delegated authority at scale.
Aviation safety tells a similar story. Autopilot systems can and do commit actions on behalf of pilots. The scope of that authority is defined with extreme precision: what the system can do, under what flight conditions, and what constraints bind it. When the autopilot acts outside its mandate, or when the mandate itself is miscalibrated, the failure is treated as an institutional failure, not a technical one. The investigation asks not only what the system did, but whether it was permitted to do it, and whether the permission was correctly specified. The governance infrastructure here is expensive, slow to evolve, and deeply conservative. That conservatism is a feature, not a bug. It reflects the cost of getting delegated authority wrong when the stakes are high.
Nuclear licensing operates on the same principle at an even more extreme end of the spectrum. The authority to operate is not assumed. It is granted explicitly, bounded precisely, and subject to continuous audit. Revocation is possible and has happened. The entire apparatus exists because the consequences of unauthorised action are irreversible and catastrophic, and because experience taught the institutions involved that implicit authority is not a viable governance model.
What makes agentic AI different from autopilot or algorithmic trading is not that it delegates authority (those systems do too), but that it delegates authority across an unbounded and expanding action space, at a speed that outpaces any existing governance cadence, and into organisational contexts where the infrastructure for bounded delegation simply does not exist. A trading algorithm operates within a well-defined market with established rules. An agentic system operates within an enterprise environment where the rules are implicit, the boundaries are social rather than technical, and the action space expands with every new tool or API connection. The governance models that worked for constrained delegation break down entirely when delegation becomes general.
There is also an economic incentive structure that actively selects against legitimate delegation. Building the infrastructure to make authority explicit, enforceable, and revocable costs time and reduces speed. Building a system that acts without that infrastructure costs less and ships faster. The market rewards the second option. This is not a market failure in the traditional sense. It is a structural feature of deploying capability before governance catches up. The institutions that will govern agentic systems do not yet have the tools to do so. The systems are already in production.
Accuracy does not grant authority
The fastest way to derail this conversation is to treat better models as the governance plan. Reduce hallucinations, improve grounding, tighten evaluation, raise reliability, and adoption will follow. It will not, because accuracy and authority are answering fundamentally different questions. Accuracy is an epistemic property. Authority is a political property. Conflating them is not an error of emphasis. It is a category mistake that most AI governance discourse has not yet corrected.
Accuracy answers whether an output is correct relative to a reference. Authority answers whether a system is permitted to take an action, in a context, on behalf of a principal, with consequences. A system can be accurate and unauthorised. It can be inaccurate and authorised. Legitimacy is not awarded by performance, and no amount of benchmark improvement changes the institutional question of who is permitted to act and under what constraints.
This is why organisations build strong pilots and still refuse to scale them. The refusal is rational. Scaling does not require accuracy first. Scaling requires bounded discretion: the ability to constrain what the system can commit, and to withdraw that permission when conditions change. An organisation that cannot do the second will not do the first, regardless of how well the pilot performs, because scaling without bounded discretion is not adoption. It is exposure.
This is also why verification remains expensive even as generation gets cheap. Verification becomes the substitute for authorisation. Organisations try to buy permission by auditing outcomes because they lack an enforceable mandate model. The bottleneck is not that verification is inherently costly. It is that verification is filling a governance hole that authorisation should have closed. The cost is not a property of the verification process. It is a property of the governance architecture.
Frameworks that matter converge on this point. NIST treats AI risk as lifecycle governance, not a one-time accuracy threshold. The OECD grounds accountability in roles and institutions, not model scores. The EU’s obligations for higher-risk systems emphasise sustained oversight and post-deployment duties. This is another way of insisting that governability must persist after launch, and that performance at launch does not settle the governance question for the lifetime of the system.
Accuracy matters. It just does not solve the hard part. The hard part is institutional: who is allowed to commit, where, under what constraints, and how authority is withdrawn. If accuracy does not grant authority, then legitimacy cannot be tested into existence. A system becomes permissible only when it operates under an explicit, enforceable mandate. No evaluation suite can produce that mandate.
Transparency does not create control
When accuracy stops being persuasive as a governance response, transparency becomes the next refuge. Make the system explainable. Log everything. Trace decisions. Monitor behaviour. Audit outcomes. Prove diligence. Transparency is a stable response because it satisfies institutional needs without changing the runtime authority model. It strengthens the ability to produce an accountability narrative without strengthening the ability to govern while the system runs. The two capabilities are not the same, and treating them as interchangeable is one of the more consequential errors in current AI governance thinking.
The distinction is simple but its implications are large. Transparency is observational. Control is interventional. A system can be completely transparent and still be illegitimate, because illegitimacy is unauthorised action. You can watch unauthorised action happen in perfect detail. The watching does not stop it.
Logs tell you what happened after it happened. Audits tell you what happened after it happened. Explanations tell you why the system claims it happened. None of these stop the next action. None narrow discretion. None withdraw permission. They produce records, not restraints. The entire transparency apparatus is oriented toward the past. Governance requires orientation toward the future: the ability to deny, constrain, or revoke before the next action commits.
Auditability is not legitimacy. An audit can be perfect and still arrive too late to matter. A decision can be fully traceable and still be indefensible if the mandate was never valid, never bounded, or never revocable. This is where accountability gets inverted. Humans get assigned monitoring and cleanup while lacking power to prevent or reverse actions. They become liability sponges: absorbing the institutional need for someone to be responsible without absorbing any of the power that would make responsibility meaningful.
Institutional frameworks signal this distinction even when they do not use this vocabulary. NIST treats monitoring and documentation as components of governance, not governance itself. The OECD links transparency to accountability for AI actors, which points back to authority structures rather than mere visibility. EU obligations for higher-risk systems imply intervention capacity, not just an audit trail. The trajectory across all three is the same: transparency is necessary but it is not sufficient, and building governance around it is an admission that the harder problem, runtime authority, has not been solved.
Oversight is not intervention
When transparency fails to produce control, organisations reach for oversight: boards, sign-offs, committees, human-in-the-loop architectures. Oversight sounds like power. Often, it is not. Oversight is a viewpoint. Intervention is a capability. Governance requires the second, and most of what passes for oversight in AI deployment delivers only the first.
A committee can review a system and still be unable to stop its actions in practice. A human can be “in the loop” and still be unable to prevent or reverse outcomes. A sign-off can be required and still be meaningless if the authority it grants is broad, persistent, and hard to revoke. The only runtime question that matters is not “was this reviewed?” It is: who can actually stop it, in time, before the next action commits?
Most “human-in-the-loop” implementations do not answer that question in the affirmative. They are throughput management or blame management dressed as governance. Only one pattern deserves the name: a governor with enforceable runtime authority to deny, override, pause, and revoke, with enforcement the system cannot route around. Everything else is a record of having looked.
Oversight rituals persist because they satisfy institutional needs without re-architecting authority. They produce records of “due process.” They preserve velocity. They relocate blame to the reviewer rather than the deployer. They do not create runtime handles. Legitimacy is not a certificate granted at deployment time. It is contextual and continuous. Oversight is batch-oriented. Runtime is continuous. This mismatch is why agentic systems escape committee governance even when every individual component was reviewed and approved. Composition produces new authority surfaces that no prior sign-off covered, and batch-oriented governance has no mechanism to address them.
NIST’s lifecycle framing points to the same requirement: governance is ongoing and connected to response, not a one-time gate. The EU’s emphasis on human oversight carries weight only under a hard interpretation: intervention capacity that can stop or override system behaviour before harm propagates. Procedure does not prevent harm at scale. If your strategy is review, approve, monitor, and escalate, you have a workflow for producing accountability narratives. You do not have governance. Governance requires runtime handles: scope enforcement per action, pauses, overrides, and revocation that propagates across the systems involved.
How power moves through systems
Power in automated systems does not announce itself. It does not arrive as a policy declaration or a governance decision. It arrives as a default. A default permission structure in an API. A vendor SDK that ships with permissive scopes because restrictive scopes require configuration that most operators will not perform. A protocol that assumes trust unless explicitly told not to. These are not neutral technical choices. They are governance commitments made at the architecture layer, and they determine who has discretion before any policy document is written.
Consider how a typical agentic deployment accumulates authority. A platform provider ships an agent framework with a set of tool integrations. Each integration carries permissions: read access here, write access there, the ability to trigger actions in a third-party system. The defaults are set by the platform provider, not by the operator who deploys the system. The operator can restrict them. Almost none do, because restriction requires understanding the full permission graph, which requires documentation that is either incomplete or written for a different audience. The result is that the agent operates with the authority the platform decided to grant, and the operator inherits that authority profile without having explicitly chosen it.
This is how power concentrates silently. The platform provider becomes, in practice, the governor of what the agent can do, even though the operator is contractually responsible for what it does. The beneficiary is the platform provider, whose product appears capable and whose users ship faster. The loss-bearer is the operator, who inherits liability for actions it did not explicitly authorise. The governor, the entity with actual runtime control, is the platform, and it has no incentive to make that control visible or transferable.
Economic incentives reinforce this structure. Speed to market rewards platforms that ship permissive defaults. Cost of friction penalises operators who invest in permission hygiene. The organisations that would benefit most from restrictive defaults, those that bear the most liability, are precisely the ones with the least leverage to demand them, because they are buying capability, not selling it. The market does not correct this. It selects for it.
Standards and protocols compound the problem. When an industry converges on a protocol that assumes broad permissions by default, as much of the current API ecosystem does, the governance implication is locked into the infrastructure. Changing the default requires changing the standard, which requires consensus across the entities that benefit from the current default. This is not a bug in the standards process. It is a feature of how standards encode power. Early decisions about what is permitted by default become institutional facts. They harden. They resist change. And they determine, for years or decades, who has discretion and who does not.
Registries are another mechanism through which power concentrates invisibly. A registry determines what is discoverable. In the context of agentic systems, the registry of available tools, APIs, and integrations determines what an agent can reach. The entity that controls the registry controls the action space. If an action is not in the registry, the agent cannot commit it. If an action is in the registry, the agent can attempt it, subject only to whatever permission checks the action’s endpoint performs. The registry is, in this sense, a governance mechanism. But it is almost never designed as one. It is designed for discoverability and developer experience. The governance implications, including who can act, on what, and whether anyone is watching, are not part of the design specification. They are side effects.
The lesson from prior infrastructure domains is consistent. In financial regulation, the entities with the most to lose from uncontrolled authority delegation, investors rather than brokers, had to organise politically to change the defaults. In aviation, the defaults around autopilot authority were changed only after failures made the cost of the existing defaults visible. Power does not redistribute itself. It is redistributed when the cost of the current distribution becomes undeniable.
Integration is where authority breaks
Integration is where organisations feel justified calling this “engineering.” APIs, pipelines, access control, architecture diagrams. And yet integration is where authority becomes illegible by default, because most enterprises do not integrate systems. They integrate exceptions. They stitch together legacy processes, vendor platforms, policy workarounds, and organisational compromises into something that works as long as humans are the glue. Then they introduce agents, which act like perfect glue: faster routing, more tool calls, more composed steps, more side effects. The agents accelerate throughput. They also accelerate authority migration across boundaries without authority being re-specified.
The adoption bottleneck at integration is not a capability gap. It is a mandate gap: the inability to say, in enforceable terms, what actions are permitted, under what conditions, and which actor is accountable for each commitment. An organisation may have clear authority structures for each individual system in isolation. The moment those systems are connected through an agent, the authority structures do not compose. They collide. And the collision produces new authority surfaces that none of the original structures anticipated.
Integration breaks legitimacy in three predictable ways. First, it collapses roles. A system that was a data source becomes an actuator because a connected agent can trigger writes through it. A tool that was read-only in practice becomes action-capable because the norms that constrained it are social, not technical, and the agent does not observe social norms. The controller role expands because connectivity exists, not because anyone granted expanded authority.
Second, integration hides the governor. Constraints become distributed, inconsistent, and often uninspectable. Some systems enforce thresholds. Some do not. Some support revocation semantics. Some require manual cleanup. Some vendor controls are opaque to the operator. The result is partial brakes and no master switch. The governor, if one exists, cannot see the full authority graph, and therefore cannot enforce it.
Third, integration launders liability. Authority dissolves into interfaces. Accountability dissolves into contracts. Everyone owns a link in the chain, nobody owns the chain itself, and the loss-bearer remains downstream while discretion remains upstream or diffuse. This is not an accident of poor contract drafting. It is a structural feature of how integration distributes power across boundaries that were not designed to be crossed by automated systems.
This is also where architecture becomes destiny in the most consequential sense. The decision to use a permissive API rather than a restrictive one, the decision to integrate through a shared service bus rather than an isolated connector, the decision to store credentials in a form that an agent can use without re-authorisation. Each of these is a technical choice. Each of them is also a governance commitment, and each of them hardens into an institutional fact the moment it is deployed. Reversing it requires not just a technical change but a renegotiation of the authority relationships that the architecture encoded. Most organisations do not recognise this until the authority relationships have already produced consequences.
If an integration project cannot answer, for each committed action, who is the controller, who is the governor, who verifies conditions, who benefits, and who bears loss, then the integration has not produced capability. It has produced exposure. Assistance tolerates illegible authority because humans still commit and humans still absorb blame. Agency cannot tolerate illegible authority because it turns integration into commitment at speed and scale, and illegible commitment at scale is not a governance gap. It is a governance absence.
Failure and harm as throughput
The standard way to discuss AI failure is to treat it as an accident: an exceptional event that a better system would have prevented. This framing is almost always wrong. In systems where authority is illegible, failure is not exceptional. It is structural. It is produced by the architecture itself, at a rate proportional to throughput. The more actions the system commits, the more failures it produces, not because it is malfunctioning but because the authority under which it acts was never adequately specified.
Adversarial exploitation is the most visible failure mode, but it is not the most consequential. An adversary who understands that a system’s mandate is ambiguous can engineer actions that fall within the ambiguity. They can structure requests that the system is technically permitted to fulfill but that no principal actually authorised. The exploit is not in the model. It is in the mandate. The ambiguity that an adversary exploits is the same ambiguity that a well-intentioned system exploits when it exercises discretion nobody bounded. The adversary simply makes the exploitation visible.
Silent exclusion is harder to detect and more consequential in aggregate. When a system commits actions on behalf of an institution, it does so according to the scope it was given. That scope was defined by whoever configured the system, typically someone inside the institution, with the institution’s interests as the reference point. The affected parties, the people on whom the institution’s actions bear, were not consulted in the scoping. They are not represented in the mandate. They have no standing to contest decisions made under it. The system routes around them not because it is malicious but because the architecture provides no mechanism to include them. Harm is produced not by error but by the absence of contestation infrastructure.
Irreversibility is the failure mode that makes all the others dangerous. Many actions that automated systems commit are not reversible in any meaningful sense. A message sent. A payment processed. A record created. A decision logged and acted upon downstream. The action may be technically undoable, but undoing it requires knowledge of what it triggered, authority over the systems it affected, and speed that the governance infrastructure does not support. When the architecture does not distinguish between reversible and irreversible actions, when all actions are treated as equivalent in terms of the authority required to commit them. Irreversible actions proceed at the same rate as reversible ones. The harm is not an edge case. It is baked into the throughput.
Lock-in compounds all of these. Once an authority structure is encoded in an integrated system and actions have been committed under it, changing the structure requires not only a technical modification but a renegotiation of every downstream consequence. Organisations that discover their authority architecture is producing harm often find that correcting it would require unwinding months of committed actions, renegotiating vendor contracts, and re-specifying permissions across systems that have already been built to assume the current defaults. The cost of correction becomes prohibitive. The illegitimate structure persists not because anyone defends it but because the alternative is more expensive than the harm.
The rate at which harm accumulates is therefore not a function of system quality. It is a function of throughput and mandate clarity. A system that commits ten thousand actions per day under an ambiguous mandate produces ten thousand opportunities for illegitimate outcomes per day. The system does not need to malfunction to produce harm. It needs only to act within the ambiguity it was given. Organisations that frame this as a reliability problem are looking at the wrong variable. The variable that controls harm rate is not the error rate of the model. It is the specificity of the mandate.
Human-in-the-loop does not bound liability
After accuracy fails to produce governance, transparency fails to produce control, and oversight fails to produce intervention, one phrase survives as a final defence: human-in-the-loop. It survives because it sounds like a compromise. A human somewhere in the workflow; therefore, accountability is preserved. Human-in-the-loop does not bound liability unless the human is a governor with enforceable runtime authority. Most implementations do not meet that condition. They are throughput mechanisms or liability mechanisms. They make the institution feel safer without making the system more legitimate.
The failure is structural, not incidental, and it operates through four reinforcing mechanisms. Under load, the human collapses into a rate limiter. Throughput pressure converts review into default approval. The veto becomes socially unavailable because using it consistently would halt the workflow that everyone depends on. The human is still “in the loop” in a technical sense. They are not governing in any meaningful sense.
In integrated systems, the human cannot see the full authority chain. They are asked to approve outputs without being shown the conditions that make an action legitimate or illegitimate. The loop becomes performative because the system was never designed to present mandates in checkable form at decision time. The human approves what is visible. The authority that matters is invisible.
The human is also treated as a substitute for revocation. Monitoring and escalation are not revocation. A human can request that authority be withdrawn. They cannot make revocation propagate across distributed tools unless the infrastructure for propagation exists. Without revocation semantics, humans become witnesses, not governors. They observe the system’s behaviour. They report on it. They cannot change it.
The deepest problem is that human-in-the-loop shifts responsibility without shifting power. The controller keeps acting. The beneficiary keeps capturing upside. The loss-bearer keeps absorbing harm. The human becomes the legitimising layer that allows institutions to claim accountability without creating control. That is why the phrase is politically useful. It is why regulators accept it and operators deploy it. It is not why it works, because it does not work as governance. It works as a narrative.
Contestability is not optional
A mandate can be valid and still produce illegitimate outcomes. Validity is a property of the mandate’s internal structure: does it bind the right actor, scope the right actions, specify the right conditions? Legitimacy is a property of the mandate’s relationship to the people it affects: can they see it, understand it, and challenge it? A mandate that is technically valid but not contestable is not legitimate. It is arbitrary.
Contestability requires standing. Standing is the right to challenge a decision, and it is not automatically granted. In most automated systems, standing is determined by the architecture. The entities that can see the mandate are the entities that configured it. The entities that can challenge it are the entities that have access to the system’s governance layer. Affected parties, the people who bear the consequences of the system’s actions, typically have neither access nor standing. The architecture makes contestation structurally impossible for the people who need it most.
This is not a failure of policy or intent. It is a failure of design. The systems are designed to serve the deployer. The governance apparatus is designed to satisfy the deployer. The contestation path, if it exists at all, routes through the deployer. When the deployer is also the beneficiary, meaning the entity that profits from the system’s actions is the same entity that controls the contestation path, contestability collapses into self-review. The system is judged by the entity that benefits from its continued operation.
Remedy is the other half of contestability. Even when a challenge succeeds, when an affected party demonstrates that an action was illegitimate, the question of what can be done about it is constrained by the architecture. Some actions are reversible. Most are not. Compensation can be offered, but it does not undo the action. The remedy path must be designed into the system before actions are committed, not bolted on afterward. An architecture that does not distinguish reversible from irreversible actions, and does not build remedy paths for the irreversible ones, has made a governance decision: it has decided that harm is acceptable as long as it is efficient.
What contestability looks like when it works is not a complaints process or a feedback form. It is a structural property of the mandate itself. The mandate specifies not only what the system can do but who can see that it is doing it, under what conditions that visibility is triggered, and what standing affected parties have to intervene before the action completes. This is not a post-hoc mechanism. It is a pre-commitment: the system was designed with the assumption that its actions will be challenged, and the challenge path was built into the architecture before the first action was committed. Organisations that have achieved this in other domains, such as financial exchanges with real-time surveillance and regulated utilities with public intervention rights, did so because the alternative was politically and economically unacceptable. The alternative for agentic systems is becoming unacceptable at a rate that most organisations have not yet registered.
EU obligations for higher-risk systems move in this direction, emphasising post-deployment duties and the ability to intervene after deployment. But the trajectory required is more radical than current frameworks acknowledge. Contestability is not an add-on to a system that was designed without it. It is a structural property that must be present from the point at which the mandate is specified. If it is not, the mandate is not legitimate, regardless of whether it is technically valid and regardless of whether the system performs well.
The minimal structure that makes legitimacy possible
After the eliminations of accuracy, transparency, oversight, and human-in-the-loop as governance strategies, the temptation is to swing into solutions: frameworks, maturity models, best practices. That would be a category error. This essay is not arguing that organisations should improve governance incrementally. It is arguing that most organisations do not yet have governance in the only sense that matters under automation: the ability to grant, constrain, and withdraw authority while the system runs. The constructive move stays minimal. Not comprehensive. Just the smallest structure that makes legitimacy possible.
Legitimacy under automation requires executable mandates. Not policies. Not intent. Not documentation. Mandates evaluated at the point of action, enforced by the system, and revoked in a way that actually propagates. A mandate is the unit of delegated authority. If it is not machine-checkable and revocable, it is not a mandate. It is a story that the institution tells itself about what it has authorised.
An executable mandate binds five things together in enforceable form. A named actor prevents service-account ghosts: the diffuse, unattributable authority that accumulates when systems act without a traceable principal. A scope of permitted actions separates access from authority; having access to a system does not mean being permitted to commit actions through it. Conditions under which those actions are permitted move governance from norms into evaluation at decision time: the system checks whether the conditions hold before it acts, not after. Evidence required to act prevents legitimacy from resting on internal assertions. The system must be able to demonstrate, not merely claim, that the conditions for action are met. A revocation path that works in practice converts control from meetings into operations: the decision to withdraw authority propagates to the systems that enforce it, without requiring a human to manually reconfigure each one.
Each element maps onto a verifiable attestation: something a system can check at runtime, not merely something a person can describe in a policy document. This is why many organisations discover their governance stack is optimised for narration. They can reconstruct what happened. They cannot stop what happens next. They can approve deployments. They cannot bound actions at runtime. They can escalate. They cannot revoke at speed. The gap between what they can narrate and what they can enforce is the gap between governance theater and governance.
Now add two properties required for legitimacy in practice: contestability and remedy. A mandate that cannot be challenged by affected parties is arbitrary. A mandate that cannot produce remedy when it produces harm is extractive. A legitimate mandate therefore implies a contest path (who can challenge, how quickly, with what standing), a remedy path (what can be reversed, what cannot, what compensation exists when reversal is impossible), and a governor (who can pause or override, and how that authority is exercised in the system, not just in policy). Integration keeps producing authority sprawl for precisely this reason: integration is the movement of mandates across boundaries, and if mandates are not enforceable across systems, integration creates controllers everywhere and governors nowhere.
What this rules out
An argument earns its place by being explicit about what it rules out. This one rules out several positions that are currently popular in AI governance discourse, and it does so not by disagreeing with them on values but by showing that they are insufficient given the structural properties of automated systems.
It rules out the position that governance can be achieved through better documentation. Documentation is narration. It describes what was intended. It does not enforce what is permitted. An organisation with perfect documentation and no enforceable mandate model is no more legitimate than one with no documentation at all, because documentation does not constrain runtime behaviour. The gap between what is documented and what is enforced is not a quality problem. It is an architectural one.
It rules out the position that voluntary industry standards are sufficient. Standards are only as strong as their enforcement mechanism. A standard that recommends enforceable mandates but does not require them produces compliance theater: organisations demonstrate adherence to the standard without acquiring the capability the standard is meant to produce. The entities that most need enforceable mandates, those that bear the most liability, are the ones with the least incentive to adopt them voluntarily, because adoption costs speed and flexibility. Voluntariness selects against the organisations that need the governance most.
It also rules out the position that self-regulation by AI developers is an adequate governance mechanism. Self-regulation works when the regulating entities bear the consequences of under-regulation. AI developers do not bear the primary consequences of illegible mandates. The operators who deploy the systems bear them. The users and frontline staff affected by the systems bear them. The developers capture the revenue. Self-regulation in this structure is not a governance mechanism. It is a competitive advantage dressed as responsibility. The entities that would need to be governed are the ones making the governance rules.
It rules out the position that the problem is primarily technical and will be solved by better AI systems. Better systems can be more accurate, more interpretable, more aligned with stated intent. None of these properties grant authority. A perfectly aligned system that acts without a revocable mandate is still illegitimate. The problem is not in the system. It is in the relationship between the system and the institutional structures that are supposed to govern it.
The argument also has costs, and intellectual honesty requires naming them. Enforceable mandates slow things down. They introduce friction at the point of action. They require organisations to do the hard work of specifying authority explicitly, which is expensive and which many organisations will resist because explicit authority makes power visible and visible power is contestable. The entities that currently benefit from illegible authority, including platforms that ship permissive defaults, operators that avoid liability through ambiguity, and beneficiaries that capture upside without bearing downside, will resist enforceable mandates because enforceable mandates constrain them. This is not an argument against enforceable mandates. It is a prediction about who will fight them and why.
There is also a real tradeoff between mandate specificity and adaptability. An executable mandate that is too narrow constrains the system from acting in situations that the mandate designer did not anticipate. An executable mandate that is too broad permits actions that no one intended. The sweet spot is not a fixed point. It moves with context, and finding it requires continuous governance, not one-time specification. This is why revocation is not an optional feature of the mandate model. It is the feature that makes the model survivable in practice. Without revocation, mandates become permanent allocations of authority. With revocation, they become adjustable constraints.
The ceiling is institutional
There is a reason AI keeps getting deployed where the stakes are low and the authority is soft. Most institutions do not yet have a way to make delegated authority legitimate at runtime. So the equilibrium holds. Assistance scales because it does not require new authority. Humans still commit. Humans still absorb blame. Institutions can claim progress without re-specifying mandates.
Agency does not scale because agency is authority, and authority without revocation is exposure. The moment a system can commit actions, you need to know, mechanically, who granted that power, where it applies, what constraints bind it, who can stop it, and how harm is remedied when it slips. If you cannot answer those questions in a way the system can enforce, then your governance posture is performative. It produces narratives, not restraints.
That is why accuracy will not unlock adoption. That is why transparency will not unlock control. That is why committees will not govern runtime systems. That is why integration keeps stopping short of letting anything act. Those are all substitutes for the same missing primitive: executable mandates. Power moves through defaults, scopes, and enforcement. When those are not explicit, power moves anyway. Controllers gain discretion because automation is cheap. Beneficiaries capture upside because speed compounds. Loss-bearers inherit downside because contestability collapses. Governors are invoked rhetorically but not instantiated mechanically.
Institutions call this a trust problem because “trust” is the safest word for illegible authority. It avoids naming who has power and who is being made responsible for outcomes they cannot control. But what is happening is not mistrust. It is a rational refusal to delegate without handles.
The sorting mechanism is this. If your organisation can grant authority in machine-checkable form, enforce it at decision time, and revoke it in a way that propagates, you will scale agency. Not because you are more visionary, but because you can bound liability and preserve legitimacy under automation. If your organisation cannot do that, you will stall at assistance. You will keep investing in monitoring, audits, reviews, and procedural oversight. You will call it safety. You will call it responsibility. You will call it trust. And you will still avoid commitment, because commitment forces the question your institution cannot answer: who is allowed to act?
The uncomfortable implication is this: the organisations best positioned to solve the legitimacy problem are not the ones building the most capable systems. They are the ones that have already built governance infrastructure for delegated authority in other domains, including financial institutions, regulated utilities, and aerospace operators. The organisations building the most capable AI systems are, by structural incentive, the ones least likely to constrain that capability with enforceable mandates, because constraint reduces the speed and flexibility that make the capability valuable. The market will not self-correct. The governance infrastructure will not emerge voluntarily. The entities that bear the losses when it does not will have to organise to demand it, exactly as they did in every prior domain where delegated authority produced harm at scale.
That is the ceiling. It is not technical. It is institutional. AI will not scale until authority becomes machine-checkable, revocable, and contestable at runtime. The organisations that figure this out first will not have better models. They will have better mandates. Everything else is decoration.