AI Governance Framework: What You Actually Have to Build
Read five guides on building an AI governance framework and you will come away with pillars, principles, maturity models and a diagram with arrows. Read a sixth and you will find a different set of pillars.
What you will not find is the thing you need before you can start: a list of what the framework physically consists of. Not what it should achieve — what it is made of. Which documents exist, who owns each one, what gets recorded where, and which of those a standard actually requires rather than a consultancy recommending them.
This article is that list. It is written from producing documentation sets against ISO/IEC 42001 and the EU AI Act, and it separates three things that guidance usually blends: what a standard requires, what the law requires, and what is your own choice.
Building this rather than reading about it? The ISO 42001 Compliance Suite contains the whole set — sixteen procedures, ten record forms, nine registers and four guidance documents, with the Statement of Applicability pre-populated with all 38 Annex A controls. €590 excl. VAT.
First: governance is not a document, it is a set of decisions that leave records
The framing problem in most guidance is that it treats governance as something you write. It is not. It is a set of recurring decisions — which systems we will build, on what data, with what oversight, for whom, and what we do when one misbehaves — and the framework is the machinery that makes those decisions happen on a schedule and leave a trail.
That distinction has a practical test. For each element below, ask: what record does this produce, and how often? An element that produces no record is a statement of intent, not a control. It may still be worth having; it is not governance.
The consequence is that a framework you can buy or download is half a framework. The documents are the cheap half. The records are produced by operating them, and they take months.
The three layers
An AI governance framework has a shape, and it is the same shape whether or not you certify.
Layer 1 — the management system. The generic machinery: scope, policy, roles, objectives, competence, document control, audit, review, corrective action. If you already run ISO 9001, ISO 27001 or ISO 13485, you already own this layer and should extend it rather than build a second one.
Layer 2 — the AI-specific processes. Impact assessment, risk, the AI system life cycle, data management, transparency, human oversight, concerns and incidents, value-chain responsibility. This is what makes it AI governance rather than governance.
Layer 3 — the records. What each process produces, and what an auditor, a regulator or a customer will ask to see.
Guidance that only addresses layer 2 leaves you with processes nobody runs. Guidance that only addresses layer 1 leaves you with a management system that governs nothing in particular.
Layer 1: the management system
Nine elements. All of them exist in every management system standard, so all of them transfer if you run one already.
| Element | What it is | Record it produces |
|---|---|---|
| Scope | Which AI systems are governed, stated as a boundary | The scope statement, approved |
| Policy | What the organisation commits to on AI, signed by top management | The policy, dated and communicated |
| Roles | Who owns what: AI system owner, impact assessor, oversight reviewer | Role assignments |
| Objectives | What you are trying to achieve, with a measure and a date | Objectives register |
| Competence | What each role must be able to do, and evidence they can | Competence register |
| Documented information | How documents and records are controlled | Master document list |
| Internal audit | Independent check that the system operates | Audit programme and reports |
| Management review | The meeting where decisions get made | Minutes with decisions |
| Nonconformity | What happens when something fails | Nonconformity and corrective action register |
The one that decides whether the rest works is the scope. Most organisations draw it around the AI in their product and leave out the AI they use to run the business: models in complaint triage, in recruitment screening, in code generation, in customer support. Those are AI systems the organisation deploys, with real consequences for real people, and a framework that excludes them governs the smaller half of the problem.
The one everyone underestimates is competence. Not training attendance — evidence that the person doing the impact assessment can distinguish a consequence for the organisation from a consequence for the person affected. Those are different skills and the second one is rare.
Layer 2: the AI-specific processes
Eight processes. This is where an AI governance framework differs from any other, and where the work is.
The AI system inventory
Not a process, but everything depends on it. For each system: what it is, what it is used for, who owns it, what role you hold — provider, deployer, or both — where it operates, and the status of its impact assessment.
The role matters more than most inventories reflect, because obligations attach to roles, and the role can change without the system changing. Modify a bought system substantially, or put your own name on it, and you may have moved from deployer to provider.
Expect the count to be roughly double your first estimate. The tools people signed up for individually, the AI features switched on inside software you already licence, and the model behind a supplier’s service all count.
Impact assessment
The core of AI governance and the piece with no equivalent in any other management system.
It asks what consequences the system has for individuals, for groups of individuals and for societies — including when it works exactly as designed. Not what happens when it fails: what happens when it succeeds at what it was built to do.
A triage model that performs well across a population and worse for one subgroup has not failed. A credit model that declines applicants consistently according to its training has not malfunctioned. Both produce consequences that no failure-mode analysis surfaces.
Three rules that make the difference between a real assessment and a decorative one:
- Do it before deployment, and record the date. An assessment dated after go-live is the single most common finding in this area, and it is unarguable.
- At least one assessor who did not build the system. An assessment written only by the builders rarely finds a problem.
- Do it for low-impact systems too, briefly. The conclusion that a system is low-impact is itself a conclusion that has to be documented.
AI risk
Separate from your security risk process, and separate for a specific reason: the consequence scale is different. Security risk scores harm to the organisation. AI risk has to score consequences to people, and the two scales cannot be collapsed without the security one winning — because it is the one the organisation already knows how to use.
Score the consequence for the organisation and for the people affected separately, and use the higher of the two.
The AI system life cycle
Stage gates from requirements to retirement, with a decision at each one. What distinguishes this from a normal software life cycle is that the gates ask whether the system should proceed, not only whether it works: is the intended purpose still what we said, is the data still fit, has the impact assessment been revisited since the last material change.
Include retraining. A model retrained on new data is a change to the system, and most life cycles treat it as maintenance.
Data management
Four questions your engineering process probably does not answer in writing:
- Provenance — where did it come from
- Rights — do you have the right to use it for this purpose. Consent for a service is not consent for model training
- Quality for the purpose — assessed against the population the system will be used on, not the one it was trained on
- Bias — as a property of the data and the design, not only as a performance metric
The uncomfortable version is retrospective: a model trained two years ago on a dataset assembled for something else. Far cheaper to answer before training than after.
Transparency and information
Two audiences, and they are not the same.
Users get documentation: what the system does, the limits of valid use, what it should not be used for. People affected by outputs get something different and shorter: that a decision involved an AI system, and how to question it.
The second audience is the one nobody plans for, because they are not customers. Under the EU AI Act some of this is a legal duty rather than good practice — the Article 50 transparency obligations have applied since August 2026 and were not deferred.
Human oversight
The test is not whether a human is in the loop. It is whether that human can, in practice, contradict the system.
Define it per system, in operational terms: who reviews, what they see, what they can change, and how much time they have. Then measure it. The measurable version is the override rate — how many outputs were changed in the last hundred. If the answer is zero across a meaningful sample, the oversight is nominal.
Nominal oversight is worse than none, because the organisation relies on it.
Concerns and incidents
One channel, open to people outside the organisation as well as inside. This is the requirement that surprises most teams: an internal reporting line is not sufficient, because the people best placed to notice that a system is behaving badly are often the people it is behaving badly toward.
Protection from retaliation, anonymity where requested, and — the detail that makes it real — the organisation does not attempt to identify an anonymous reporter.
If every entry in your incident register originated inside the organisation, the external channel is not reaching anyone.
Value-chain responsibility
For every AI system you did not build entirely yourself, who answers for what: data provision, development, validation, operation, oversight, user information, incident notification, and accountability toward the affected person.
Write it down per supplier. A responsibility recorded as “theirs” with no clause in the agreement that assigns it is the gap this produces.
Layer 3: the records, and what gets asked for first
An auditor, a customer’s security questionnaire and a regulator will converge on roughly the same ten things.
- The AI system inventory, with owner and role held
- The scope statement, checked against that inventory
- The signed policy, dated and communicated
- The risk register, with consequences for people and not only for the organisation
- The Statement of Applicability, if you are working to ISO 42001 — all 38 controls decided
- Impact assessment records, dated before go-live
- Life cycle records for at least one system, gate by gate
- The information users actually see
- Concern and incident records
- Management review minutes, with decisions
If these exist, are complete and are dated across a period, the rest follows. If they do not, no amount of framework documentation compensates, because the documents prove intent and the records prove operation.
What a standard requires, what the law requires, and what is your choice
Guidance blends these three and it causes real confusion about what is negotiable.
ISO/IEC 42001 requires seventeen items of documented information across clauses 4 to 10, plus whatever your applicable Annex A controls imply. It is voluntary. Certification is a commercial decision, not a legal one.
The EU AI Act requires specific things of specific roles, and it is law. The obligations differ sharply depending on whether you are a provider or a deployer and on the system’s risk classification. Certification to ISO 42001 does not discharge them.
Everything else is your choice, and legitimately so. An AI ethics board, a model card template, a red-teaming programme, a public transparency report — these can be excellent and none of them is required by anything. Decide them on their merits, not because a framework diagram had a box for them.
The practical rule: for each element, know which of the three categories it falls into. It changes what happens when you are short of time.
A sequence that works
For an organisation starting from an existing management system:
- Inventory the AI systems, including the ones nobody thinks of as systems. A morning of asking around beats a week of documentation.
- Record the role held for each. It determines everything downstream.
- Draw the scope, and state it separately from any other management system scope you hold.
- Map what your existing system already covers. If you run ISO 27001 or ISO 13485, layer 1 largely exists — the clause-by-clause comparison shows what transfers.
- Build the impact assessment process and run it on the highest-consequence system first. This is the largest single piece of new work.
- Define oversight per system, with a measure that would reveal it if the oversight were nominal.
- Open the concerns channel, externally as well as internally.
- Add the AI items to the existing management review. Do not schedule a second one.
Three to six months to a system that produces records, for a mid-sized organisation. The documentation is weeks; the evidence is months, and no shortcut changes that.
Frequently asked questions
Do we need ISO 42001 to have AI governance? No. The standard is one way to structure it and it is voluntary. What the standard gives you is a defined scope, a certifiable baseline and a vocabulary customers recognise. You can build the same machinery without certifying.
Where do we start if we have nothing? The inventory. Every other decision depends on knowing what you have, and the count is almost always higher than expected.
How is this different from data governance? Data governance covers whether data is accurate, secure and properly used. AI governance covers the consequences of decisions a system makes using it. They overlap on the data layer and diverge everywhere else — an AI system can run on perfectly governed data and still produce differential outcomes.
Who should own the framework? Someone with authority to stop a deployment. If the owner can only recommend, the framework documents decisions that other people make, which is a different and weaker thing.
How much of this applies if we only use AI, and do not build it? Most of layer 1, and a reduced version of layer 2. You still need the inventory, the impact assessment for consequential uses, oversight, and the value-chain allocation — because obligations attach to deployers as well as providers. The life cycle and data management shrink considerably.
What is the most common failure? A framework that exists as documents and produces no records. The second most common is a scope drawn around the product, leaving the AI used internally ungoverned.
Where to go from here
The useful test for any AI governance framework is not whether it covers the right pillars. It is whether, six months from now, you can produce ten dated records showing the decisions it was supposed to make actually got made.
Build for that and the framework comes out the right shape on its own.
The ISO 42001 Compliance Suite contains the whole set described here: sixteen procedures, ten record forms, nine registers, and four guidance documents including a requirements map that shows which clause each file satisfies. Written to extend an existing management system rather than sit beside one. €590 excl. VAT.
If you are working out where you stand, the readiness checklist is organised by what produces evidence rather than by clause number. What certification involves covers the audit and the accreditation check most buyers skip. And if the EU AI Act is what brought you here, what the Digital Omnibus changed sets out which deadlines moved and which did not.