Loading...
Loading...
Most AI safety focuses on model behaviour. Institutional failure happens at the level of authority.
AI safety research has made remarkable progress on:
But a perfectly aligned, interpretable, robust AI system can still cause institutional collapse.
The missing question:
"Even if the model behaves exactly as intended, what happens to the institutions that deploy it?"
AI systems don't just perform tasks. They concentrate authority:
Moves from distributed humans to centralised systems
Concentrates in whoever controls training data and model weights
Shifts to whoever defines metrics and benchmarks
Transfers from democratic processes to technical expertise
This is not a bug in AI systems. It is a structural consequence of capability concentration.
AI capability grows exponentially. Institutional capacity to govern it grows logarithmically. The gap is where failure breeds.
The purple area represents capability that exceeds institutional capacity to govern it
A model can be perfectly aligned and still cause institutional failure:
A well-aligned model deployed without human oversight loops
Decision-making centralised to a few operators
Some actors gain decisive advantages, destabilising existing institutions
No legitimate authority to adjudicate disputes or set boundaries
"Model alignment solves the intelligence problem. It does not solve the governance problem."
Institutions can collapse from governance failure even when all technical systems work correctly.
The alternative to reactive governance is pre-governing: designing institutional architectures before capability deployment, not after.
Without institutional design, authority concentrates rapidly. Pre-governing keeps it distributed.
Above 50%: institutional governance becomes unstable. Pre-governing keeps concentration below threshold.
This work introduces COA: a framework for understanding how intelligence systems interact with institutional authority.
COA treats AI governance not as a policy problem but as an architectural one:
When viewed through an institutional lens, familiar AI governance problems become tractable:
"How do we make AI safe?"
How do we make institutions safe from capability concentration?
"How do we align AI?"
How do we preserve authority distribution?
"Who controls AI?"
How is authority legitimised and distributed?
"What rules should AI follow?"
What institutional architecture enables governance?
To be clear, this is not:
This work asks: what institutional architectures must exist for AI capability to be governed legitimately? It is engineering, not ethics.
If this resonates, there are three ways to continue—depending on what you want to understand:
If you believe:
This work will challenge those assumptions.
If you want to understand how institutions can survive AI capability concentration,
you are in the right place.