AI safety governance faces a fundamental challenge often overlooked in technical discussions: the problem of political legitimacy. Even if we design optimal safety regulations, they face implementation only if they are perceived as legitimate exercises of authority, not mere coercion. Political philosophy offers essential tools for understanding and addressing this challenge.

The Legitimacy Problem in AI Safety

When a government bans certain AI applications, restricts compute access, or mandates safety evaluations, it exercises coercive power. But coercion alone is insufficient for stable, effective governance. As political philosophers have long recognized, there is a crucial distinction between effective authority (the ability to compel compliance) and legitimate authority (the right to rule that generates obligations to obey).

This distinction matters profoundly for AI safety. Regulations imposed without legitimacy will face resistance, evasion, and eventual collapse. Tech companies may comply grudgingly while seeking loopholes. Other nations may reject external constraints as illegitimate interference. Without legitimacy, even well-designed safety mechanisms fail.

Three Approaches to Political Legitimacy

Political philosophy offers several frameworks for understanding what makes authority legitimate. Each has implications for AI safety governance.

1. Consent-Based Legitimacy

John Locke's influential view holds that political authority is legitimate only with the consent of the governed. Applied to AI safety, this suggests:

  • National regulations: Democratic processes provide implicit consent for domestic AI rules
  • Industry self-regulation: Companies consent to voluntary frameworks
  • International agreements: Nations consent to treaties and conventions

However, consent theory faces a fundamental challenge. As David Hume observed, actual states typically arise from conquest and coercion, not consent. In AI safety, major actors often oppose regulation rather than consenting to it. A consent-based theory might conclude that regulation lacks legitimacy when imposed on unwilling tech companies or nations—a conclusion that would paralyze safety efforts.

John Simmons (2001) offers a sophisticated response: distinguish between the moral justification of states (which requires only that having states is better than anarchy) and the political legitimacy of actual states (which requires consent). Perhaps AI safety governance can be morally justified even if particular actors don't consent.

2. Public Reason and Democratic Legitimacy

John Rawls proposed that political power is legitimate "only when it is exercised in accordance with a constitution the essentials of which all citizens, as reasonable and rational, can endorse." This public reason approach shifts focus from actual consent to justifiability to all reasonable persons.

For AI safety governance, this suggests:

  • Regulations must be justified by reasons all reasonable persons can accept
  • Justifications cannot rely on controversial moral or religious doctrines
  • The focus is on constitutional essentials—basic frameworks rather than every detail

This approach has advantages. It doesn't require actual consent from every regulated entity. It provides a standard for evaluating proposed regulations. But it faces challenges: what counts as "reasonable"? How do we handle deep disagreements about AI risks and benefits?

Jean-Jacques Rousseau's related democratic theory emphasizes participation rather than hypothetical consent. On this view, legitimacy arises from active citizen engagement in lawmaking. Applied to AI safety, this might require:

  • Public deliberation about AI risks and responses
  • Democratic input into regulatory frameworks
  • Transparency enabling informed participation

The democratic deficit in current AI governance—where decisions are made by small groups of experts, executives, and officials—may undermine legitimacy on this view.

3. The Service Conception of Authority

Joseph Raz's "service conception" offers a different foundation. Political authority is legitimate, he argues, when it helps subjects better comply with reasons that already apply to them. The authority serves those it governs by making it easier to do what they have reason to do anyway.

Applied to AI safety:

  • Everyone has reasons to avoid AI catastrophe (self-preservation, concern for others)
  • AI safety regulations help us better comply with these pre-existing reasons
  • Regulations are legitimate insofar as they serve this function

This view has important implications. It suggests that legitimacy depends on effectiveness—regulations that fail to improve safety lack legitimacy. It also implies that expertise matters: authorities with better understanding of AI risks can better serve the function of helping us comply with safety reasons.

However, the service conception faces a challenge from those who deny that we have strong reasons for caution. If AI safety concerns are overstated, then safety regulations don't help us comply with real reasons—we're being "served" into unnecessary restrictions.

The Coercion Problem

A central theme across theories is the relationship between legitimacy and coercion. For Kant and his followers, coercion is not merely a tool for enforcing legitimate rules—it is constitutive of political authority. Rights imply restrictions on others; governance necessarily involves coercion.

This is particularly relevant for AI safety. Effective governance will require:

  • Banning certain AI capabilities
  • Mandating safety evaluations
  • Restricting compute access
  • Enforcing transparency requirements

All of these are coercive. The question is whether this coercion is legitimate—whether it differs from mere exercise of power.

Allen Buchanan (2002) argues that legitimacy is fundamentally about the justification of coercive power. An entity "has political legitimacy if and only if it is morally justified in wielding political power." This shifts the question from consent or participation to moral justification.

International Legitimacy: The Global Challenge

AI safety governance faces a legitimacy challenge that national governance does not: there is no global sovereign with recognized authority. This raises questions that political philosophy has grappled with primarily in the context of international relations:

  • Whose authority is legitimate globally? The UN? Coalitions of democracies? Technical standards bodies?
  • Can one nation's regulations be legitimate for others? EU AI Act, US export controls
  • What justifies coercive power across borders? Sanctions, compute governance

Traditional international law emphasizes state consent and sovereignty. But AI safety may require governance that transcends consent—if a nation refuses to constrain dangerous AI development, can other nations legitimately impose constraints?

Legitimacy vs. Justice: A Crucial Distinction

Rawls emphasizes that legitimacy and justice are distinct concepts. Political institutions can be legitimate but unjust (e.g., a moderately oppressive but democratically elected government). But just institutions are necessarily legitimate.

This matters for AI safety governance strategy:

  • Minimum viable legitimacy: Governance can be legitimate even if imperfect
  • Justice as sufficient condition: Truly just AI governance would automatically be legitimate
  • Weaker demands: Legitimacy may be achievable where full justice is not

We should not let the perfect be the enemy of the legitimately functional.

Practical Implications for AI Safety Governance

1. Build Legitimacy Through Multiple Channels

Given theoretical disagreements about the source of legitimacy, prudent governance should pursue multiple strategies:

  • Consent mechanisms: Industry input, stakeholder engagement, treaty negotiations
  • Public reason: Justifications acceptable to reasonable persons across perspectives
  • Democratic participation: Public deliberation, transparent processes
  • Service provision: Demonstrate that regulations actually improve safety

2. Address the Democratic Deficit

Current AI governance largely excludes the public. Decisions are made by experts, executives, and officials behind closed doors. While this may produce technically better rules, it risks illegitimacy from a democratic perspective. Mechanisms for public input and deliberation may be necessary even when they complicate governance.

3. Distinguish Legitimacy from Effectiveness

Weber's descriptive concept—legitimacy as belief—reminds us that perceived legitimacy matters for compliance. But normative legitimacy—actual justification—matters for rightness. We should pursue both: governance that is genuinely justified and that is perceived as justified by those it affects.

4. Recognize Partial and Graduated Legitimacy

Legitimacy is not all-or-nothing. Governance can be more or less legitimate along multiple dimensions:

  • Scope: Legitimate for some decisions but not others
  • Population: Legitimate for some groups but not others
  • Degree: Somewhat legitimate rather than fully legitimate

This suggests building legitimacy incrementally, starting where it's easiest and expanding.

5. Prepare for Legitimacy Crises

Legitimacy can be lost as well as gained. AI safety governance should prepare for scenarios that might undermine legitimacy:

  • Regulatory capture: If governance serves industry rather than public
  • Effectiveness failures: If regulations fail to prevent harm
  • Overreach: If restrictions exceed what legitimacy can support
  • Exclusion: If key stakeholders are systematically left out

Open Questions

This analysis raises several questions requiring further exploration:

  • Expertise vs. democracy: How should AI safety governance balance expert knowledge with democratic legitimacy?
  • Global legitimacy: What institutional arrangements could provide legitimate global AI governance?
  • Legitimacy under uncertainty: How does deep uncertainty about AI risks affect legitimacy requirements?
  • Legitimacy and enforcement: Can illegitimate governance be justified if it prevents catastrophe?
  • Legitimacy evolution: How can governance legitimacy be built and maintained over time?

Conclusion

AI safety governance is not merely a technical challenge of designing optimal rules. It is fundamentally a political challenge of establishing legitimate authority to make and enforce those rules. Political philosophy provides essential tools for understanding this challenge.

The key insight is that might does not make right. The ability to impose AI safety regulations does not justify doing so. Legitimacy requires something more: consent, or justifiability to reasonable persons, or service to those governed, or some combination of these. Governance that lacks legitimacy will face resistance, evasion, and eventual failure—regardless of how technically optimal its rules may be.

Effective AI safety governance requires building legitimacy alongside designing rules. This means engaging with questions of political philosophy alongside questions of technical AI safety. The two are inseparable.

References

  • Buchanan, Allen (2002). "Political Legitimacy and Democracy." Ethics 112(4).
  • Rawls, John (1993). Political Liberalism. Columbia University Press.
  • Raz, Joseph (1986). The Morality of Freedom. Oxford University Press.
  • Simmons, A. John (2001). "Justification and Legitimacy." Ethics 109(4).
  • Stanford Encyclopedia of Philosophy (2023). "Political Legitimacy." https://plato.stanford.edu/entries/legitimacy/