AI safety governance requires more than good rules—it requires authority. When regulators ban dangerous AI capabilities or mandate safety evaluations, they claim the right to be obeyed. But what justifies this claim? Political philosophy distinguishes between mere power (the ability to coerce) and authority (the right to rule that generates obligations to obey). Understanding this distinction is essential for AI safety governance that works through willing compliance rather than costly enforcement.

The Authority Problem

Every regulatory system makes a claim: "You should comply because we say so." This claim goes beyond "comply or face consequences." It asserts that the directive itself generates reasons for action—reasons that would not exist without the directive.

This is the concept of political authority: the normative power to create obligations through commands. As Robert Paul Wolff puts it, "To claim authority is to claim the right to be obeyed."

For AI safety governance, authority questions arise at every level:

  • National regulators: What gives them authority over tech companies?
  • International bodies: What gives them authority over sovereign nations?
  • Technical standards organizations: What gives them authority to set binding rules?
  • Industry self-regulation: Can private bodies claim public authority?

The answers matter because authority enables compliance without coercion. An authority that must enforce every rule through punishment is merely a powerful coercer—it lacks the genuine compliance that stable governance requires.

Three Concepts of Authority

Political philosophers distinguish three related but distinct concepts of authority:

1. The Liberty to Rule

The most minimal concept: an institution is justified in issuing and enforcing directives. This doesn't necessarily mean anyone is obligated to obey—only that the institution doesn't violate anyone's rights by ruling.

For AI safety, this might mean: regulators are permitted to create and enforce AI safety rules, without implying that companies must obey for any reason beyond avoiding punishment.

2. The Normative Power to Impose Duties

A stronger concept: the institution can create obligations through its directives. When it says "do X," subjects acquire a duty to do X—not merely a prudential reason to avoid punishment.

For AI safety, this would mean: companies have a moral duty to comply with regulations, not just a practical incentive.

3. The Right to Rule

The fullest concept: the institution has both the liberty to rule and the normative power to impose duties, plus a claim-right against interference. Subjects have correlating duties to obey and not interfere.

For AI safety, this would mean: regulators have a comprehensive moral right to govern, which others must respect and support.

Each concept has different implications for AI safety governance. The first allows rulemaking without claiming obedience; the second claims obedience without claiming support; the third claims the full package of ruling rights.

The Nature of Authoritative Directives

What makes a directive authoritative rather than merely advisory? Two key features:

Content-Independence

Authoritative directives provide reasons based on their source, not their content. "Because the regulator said so" is supposed to be a reason, independent of whether the directive is good policy.

This has important implications for AI safety:

  • Expertise questions: Should companies comply with regulations they believe are technically misguided?
  • Good faith compliance: Does authority require accepting rules one disagrees with?
  • Scope limits: Are there domains where authority doesn't generate content-independent reasons?

Pre-emptive Force

Joseph Raz argues that authoritative directives don't just add to the balance of reasons—they pre-empt competing reasons. To obey is to "surrender one's judgment" to the authority on the matters it governs.

For AI safety, this raises difficult questions:

  • Competing reasons: Should regulations pre-empt reasons like "this would help us beat competitors" or "this would advance AI capabilities"?
  • Scope of pre-emption: Do safety regulations pre-empt all competing reasons, or only some?
  • Moral limits: Are there reasons (like preventing harm) that regulations cannot pre-empt?

Theories of Legitimate Authority

When is authority justified? Several competing theories offer answers.

Consent Theory

The classic view: authority requires the consent of the governed. No one has the right to impose duties on you without your agreement.

Strengths for AI safety:

  • Respect for autonomy: companies and nations choose to accept governance
  • Clear justification: consent creates clear moral obligations
  • Limitation principle: consent can be withheld, limiting authority's scope

Challenges for AI safety:

  • Non-consent: Most AI companies don't consent to regulation; they resist it
  • Collective action: Even if individual consent were possible, how would it work for industry-wide governance?
  • International dimension: How would consent-based authority work for global AI governance?

The consent theorist's conclusion might be "philosophical anarchism": no actual AI governance has legitimate authority because none has secured proper consent. This doesn't mean no governance is possible—only that no governance can claim the full authority of consent.

Tacit Consent

Perhaps consent can be implied rather than expressed. By operating in a jurisdiction, companies tacitly consent to its regulations.

Strengths for AI safety:

  • Practical: allows for governance without explicit consent from each actor
  • Recognizes benefits: companies benefit from stable regulatory environments

Challenges for AI safety:

  • Hume's objection: the costs of rejecting governance (leaving the market) are too high for consent to be meaningful
  • Interpretation problem: how do we know what someone has tacitly consented to?
  • Global operations: which jurisdiction's rules has a globally-operating company tacitly consented to?

Functionalist Theory

Authority is justified when necessary for morally mandatory ends—security, justice, protection of rights, democratic governance. The state doesn't need consent because some goods are too important to hold hostage to individual choice.

Strengths for AI safety:

  • Catastrophic stakes: preventing AI catastrophe is a morally mandatory end
  • Collective action: functionalist theory handles coordination problems well
  • Non-consent: authority exists even over those who withhold consent

Challenges for AI safety:

  • Particularity problem: even if we need AI governance, why this particular governance structure?
  • Democratic deficit: functionalist authority might justify expert rule over democratic preference
  • Overreach: what limits exist on functionalist authority?

The particularity problem is especially relevant: if the justification for AI safety authority is preventing catastrophe, and multiple governance structures could serve this end, why must any particular actor submit to any particular regulator?

The Principle of Fairness

When you benefit from a cooperative scheme that others contribute to, fairness requires you to contribute your fair share. This generates a duty to comply with the scheme's rules.

Strengths for AI safety:

  • Mutual benefit: AI companies benefit from a stable regulatory environment
  • Collective action: safety requires everyone's participation
  • Non-acceptance: even unwilling beneficiaries may owe contributions

Challenges for AI safety:

  • Unwelcome benefits: what if companies don't want the "benefits" of regulation?
  • Alternative contributions: can companies contribute to safety in other ways (e.g., voluntary standards) instead of regulation?
  • Negligible contribution: does any single company's compliance actually matter?

Political Autonomy and Affirmation

A more recent approach, developed by Anna Stilz, holds that legitimate authority requires not consent per se, but "affirmation"—alignment between governing institutions and the values and priorities of the governed.

This matters because:

  • Without alignment: "substantial aspects of one's life can come to seem hostile, threatening, and completely beyond one's grasp"
  • Agency protection: governance that doesn't reflect subjects' values undermines their sense of agency
  • Democratic insufficiency: even democratic rule might lack affirmation if minorities are systematically overruled

For AI safety, this suggests:

  • Stakeholder input: AI companies, researchers, and affected communities should have meaningful input into governance
  • Value alignment: governance should reflect shared values about safety, not impose external values
  • Minority protection: even unpopular actors (like frontier AI companies) deserve some affirmation of their interests

Implications for AI Safety Governance

1. Authority Requires Justification, Not Just Power

The ability to enforce rules doesn't establish authority. AI safety governance must be able to explain why its directives generate obligations, not just prudential reasons to avoid punishment.

This matters because:

  • Enforcement is costly and imperfect
  • Voluntary compliance enables lighter-touch governance
  • Authority survives gaps in enforcement; mere power does not

2. Different Levels May Have Different Authority Sources

National regulators might claim functionalist authority (preventing catastrophe is a mandatory end). International bodies might rely more on consent (treaty-based) or fairness (all nations benefit from global safety). Technical standards bodies might claim expertise-based authority.

Each level should understand and be able to articulate its authority basis.

3. The Scope of Authority Matters

Even legitimate authority has limits. Raz notes that authority typically claims to pre-empt reasons only within its domain. AI safety regulators should be clear about:

  • What domains their authority covers
  • What reasons their directives pre-empt
  • What limits exist on their authority (moral limits, procedural limits, scope limits)

4. Address the Particularity Problem

Even actors who accept the need for AI safety governance might reasonably ask: "Why this particular governance structure?" Governance should be able to answer this question—not just "we need governance" but "you specifically should comply with this specific structure."

Possible answers include:

  • Legitimate process: this governance structure was created through proper procedures
  • Comparative advantage: this structure is better than alternatives
  • Practical necessity: this is the only viable structure available
  • Fairness: others are complying, so should you

5. Build Affirmation, Not Just Compliance

Stilz's analysis suggests that stable authority requires affirmation—alignment between governance and governed. AI safety governance should:

  • Include stakeholders in governance processes
  • Reflect shared values about safety and progress
  • Enable agency rather than merely constraining action
  • Accommodate reasonable disagreement about means while maintaining core commitments

6. Plan for Non-Authority

What if no AI safety governance can claim full legitimate authority? This might be true if:

  • Consent is lacking and functionalist justification is disputed
  • The particularity problem cannot be solved
  • Affirmation cannot be achieved across diverse actors

In this scenario, governance should:

  • Focus on the liberty to rule rather than the right to rule
  • Rely on enforcement rather than claimed authority
  • Build toward authority through demonstrated legitimacy and trustworthiness
  • Accept limits on what governance can demand

The International Challenge

International AI safety governance faces special authority challenges:

  • No global sovereign: there is no world state with recognized authority
  • Sovereign equality: all nations claim equal authority over their territory
  • Non-intervention: international norms restrict external authority
  • Diverse values: different nations have different views on AI risks and benefits

Possible authority sources for international AI governance:

  • Treaty consent: nations consent through treaty agreements
  • Customary law: norms become authoritative through widespread acceptance
  • Functional necessity: global coordination is necessary for safety
  • Delegation: nations delegate authority to international bodies

Each has limitations. Treaty consent doesn't bind non-parties. Customary law develops slowly. Functional necessity faces the particularity problem. Delegation is limited to what nations can legitimately delegate.

Conclusion

AI safety governance is not just about designing good rules—it's about establishing legitimate authority to make and enforce those rules. Political philosophy reveals that authority is complex: it involves the liberty to rule, the power to impose duties, and the right to rule. Each requires different justification.

Key lessons for AI safety governance:

  • Power is not authority: the ability to coerce doesn't create obligations
  • Authority needs justification: consent, functional necessity, or fairness
  • Scope matters: even legitimate authority has limits
  • Particularity is a problem: actors can reasonably ask "why this governance structure?"
  • Affirmation enables stability: governance aligned with subjects' values is more stable
  • Plan for partial authority: governance should work even when full authority is lacking

Effective AI safety governance requires not just technical expertise in AI risks, but political philosophy expertise in authority. The two domains are inseparable.

References

  • Raz, Joseph (1986). The Morality of Freedom. Oxford University Press.
  • Simmons, A. John (2001). Justification and Legitimacy. Cambridge University Press.
  • Stilz, Anna (2019). Territorial Sovereignty: A Philosophical Exploration. Oxford University Press.
  • Wellman, Christopher (2001). "Toward a Liberal Theory of Political Obligation." Ethics 111(4).
  • Wolff, Robert Paul (1970). In Defense of Anarchism. Harper & Row.
  • Stanford Encyclopedia of Philosophy (2023). "Authority." https://plato.stanford.edu/entries/authority/