Chapter 70 — AI governance and compliance
In one sentence
Governance is the paperwork and process that let an organisation — and a regulator — verify an AI system does what it claims and doesn't cause the harms it's supposed to avoid; it operates at a different altitude from Part 14's runtime guardrails, and increasingly it has legal teeth.
The problem
Part 14 built runtime defences for one agent: input filtering, action guardrails, output checks. That answers "can this specific request cause harm right now?" Governance answers a different set of questions, asked before a system ships and after it's running: What risk category is this system in? What documentation must exist? Who signed off on it? What happens when it fails in production — is there a paper trail? These questions have answers with legal consequences in a growing number of jurisdictions, and they don't disappear just because your prompt injection defences are solid.
The EU AI Act: a risk-tiered structure
The EU AI Act entered into force on 1 August 2024 and applies in stages: GPAI obligations began on 2 August 2025, transparency requirements on 2 August 2026, Annex III high-risk rules on 2 December 2027, and product-embedded high-risk rules on 2 August 2028. It is the most comprehensive framework as of this writing, and many other jurisdictions' proposals borrow its shape. It sorts AI systems into tiers by risk:
| Tier | Examples | Obligation |
|---|---|---|
| Unacceptable risk | Social scoring, manipulative subliminal techniques, real-time biometric surveillance in public spaces (narrow exceptions), emotion recognition in workplaces/schools | Banned outright |
| High risk | Hiring/employment decisions, credit scoring, law-enforcement tools, critical infrastructure control, medical devices, education admissions | Conformity assessment, risk-management system, data governance, technical documentation, human oversight, logging, accuracy/robustness testing — before market entry |
| Limited risk | Chatbots, deepfakes, AI-generated content | Transparency only: disclose that the user is talking to an AI; label synthetic content |
| Minimal risk | Most other AI (spam filters, recommender tweaks) | No mandatory AI Act obligations beyond generally applicable law |
General-purpose AI (GPAI) models — i.e. foundation models/LLMs — sit in a separate track regardless of tier: providers must maintain technical documentation, a summary of training data, and a copyright policy. Models deemed to carry systemic risk (roughly, trained above a very large compute threshold) face extra obligations: adversarial testing, incident reporting, and cybersecurity measures — practically, this is Part 14's guardrail work plus Part 15's adversarial-test suite (Chapter 62), but documented and reportable rather than just run once internally.
Penalties scale with severity: up to €35M or 7% of global annual turnover for unacceptable-risk violations, lower tiers for lesser breaches.
NIST AI Risk Management Framework: a process, not a law
The US NIST AI RMF (voluntary, but widely adopted as a de facto standard, including by regulators referencing it) organises governance into four functions instead of risk tiers:
- Govern — policies, accountability, and culture around AI risk, set before any specific system exists.
- Map — identify context, intended use, and foreseeable risks for a specific system (this is where you'd classify Chapter 62's support agent: what could go wrong, who's affected).
- Measure — quantify those risks with the same tools Part 11 already taught: eval sets (Chapter 49), LLM-as-judge, adversarial test suites.
- Manage — respond to identified risks: mitigate (add a guardrail), transfer (insurance, contracts), or accept (with documented sign-off).
The practical difference from the EU AI Act: NIST doesn't tell you which systems must comply or set penalties — it gives you the process to run, which you can point to as evidence of due diligence regardless of which jurisdiction's law you're actually subject to.
Model cards and datasheets: the artifact, not just the process
A model card (Mitchell et al., 2019) is a short, structured document answering: intended use and out-of-scope use, the training data (at a summary level), evaluation results (including across subgroups, not just aggregate accuracy), known limitations, and ethical considerations. A datasheet for a dataset (Gebru et al., 2018) is the equivalent for the data itself: how it was collected, whether it contains personal data, what it's not representative of.
Minimal model-card template you can actually use:
# Model Card: [system name]
## Intended use
- What it's for. What it's explicitly NOT for.
## Inputs and outputs
- What goes in, what comes out, in plain language.
## Training / grounding data
- Where the knowledge comes from (fine-tuning data, RAG corpus, or "none —
base model only").
## Evaluation
- Eval set size and source (Chapter 49). Metrics. Known failure modes found
during adversarial testing (Chapter 62, Step 8).
## Limitations
- What it will confidently get wrong. What it should never be trusted for.
## Human oversight
- Where a human approves before action (Chapter 60's action guardrails).
Who is accountable when it's wrong.
## Contact / incident reporting
- Where to report a problem, and what happens next.
What an audit actually asks for
Whether internal (pre-launch, Part 16's spirit), third-party, or regulatory, an audit wants evidence, not assurances:
- The model card and any datasheets.
- The risk classification (which tier, and why).
- Eval results over time, not just at launch (Chapter 55's monitoring).
- The guardrail configuration itself (Chapter 60's layered defence) — what's enforced in code, not just described in a prompt.
- The adversarial test suite and its results (Chapter 62, Step 8) — an auditor treats "we didn't test for X" as a finding.
- An incident log: what went wrong in production, when, and what changed afterward.
- Sign-off records: who approved deployment, and against what criteria.
Worked example: reclassifying Chapter 62's support agent
Chapter 62 built a support agent that looks up orders and processes refunds with human approval. Two governance readings of the same system:
- As shipped (customer service, refunds under a human-approval gate): EU AI Act limited risk — mainly a transparency obligation (tell the user they're talking to an AI). Its own guardrails already exceed what's legally required here.
- If it also decided credit limits (a small change: let it approve or
deny a credit increase instead of just a refund): this crosses into high
risk (credit scoring) — now conformity assessment, documented risk
management, and human oversight are legal requirements, not optional
hardening. The
guarded.pyexecutor and human-approval gate from Chapter 62 stop being "good practice" and become the compliance mechanism an auditor will specifically ask to see.
The lesson: the same engineering (Part 14 and Part 15) can satisfy both a security goal and a compliance goal — but which regulatory tier you're in depends on what the agent is allowed to decide, not on how well it's built.
Common confusions
- Governance is not security. A system can pass every prompt-injection test in Chapter 62's adversarial suite and still be non-compliant — for example, having no logged human-oversight record, or no risk assessment on file. They're complementary, not substitutes.
- Compliance is not a one-time checkbox. The EU AI Act requires post-market monitoring for high-risk systems — the same continuous evaluation Chapter 55 already argued for on purely engineering grounds now has a legal reason too.
- "We used a guardrail framework" is not documentation. An auditor wants to see your risk assessment and your eval results for your system, not a vendor's marketing claim about their tool.
Practice (30 min)
Write a one-page model card for Chapter 62's support agent using the template above. Then answer: which EU AI Act tier does it sit in as described in Chapter 62, and what would have to be added if it were extended to approve credit limit increases?
Check yourself
- What's the difference in what Part 14 guarantees vs. what governance requires?
- Name the EU AI Act's four risk tiers and one example system in each.
- What are the four functions in the NIST AI RMF, and which one maps most directly to Chapter 49's eval sets?
- Why might the exact same codebase sit in two different EU AI Act risk tiers depending on what it's used for?
Further reading
- Regulation: the EU AI Act — official text and the European Commission's plain-language summary.
- Framework: NIST AI Risk Management Framework (AI RMF 1.0), nist.gov.
- Paper: Mitchell et al., "Model Cards for Model Reporting" (2019, arXiv:1810.03993).
- Paper: Gebru et al., "Datasheets for Datasets" (2018, arXiv:1803.09010).
- Reference: OWASP LLM Top 10 — an engineering-facing companion to these governance frameworks (already in Chapter 63's gap table).
End of Part 18 — Milestone check
- I can name three concrete things an agent framework gives you that a raw loop doesn't, and I know none of them improve the model's reasoning.
- I re-expressed a hand-written agent loop as a small LangGraph graph.
- I can place a given AI system into one of the EU AI Act's four risk tiers and justify why.
- I wrote a model card for a system I built in this guide.