How to Run an AI Risk
Assessment: A Framework for
Security Teams

EU AI Act Article 9 requires a risk management system across the entire lifecycle of every high-risk AI system. It does not tell you how to build one. NIST's AI RMF tells you what good looks like across 72 subcategories, but issues no certification and no scoring method. This is the working process that fills the gap — scope, score, register, treat, monitor.

Art. 9
mandates a lifecycle risk management system for high-risk AI — and deliberately doesn't prescribe the method
12
generative-AI risk categories NIST identifies that classic IT risk assessments never cover
72
subcategories across the AI RMF's four functions — why you scope an assessment instead of crawling a checklist
0
certifications the AI RMF offers. It's voluntary — so your assessment must generate its own evidence

Most security teams already know how to run a risk assessment. You inventory assets, identify threats, estimate likelihood and impact, score, prioritize, treat, and monitor. The methodology is decades old and it works. So the natural instinct when AI arrives is to point that same machinery at the new systems and call it done.

That instinct is half right. The structure of risk assessment transfers cleanly. What doesn't transfer is the threat model, the failure modes, or the unit of analysis. A traditional assessment asks whether an attacker can compromise a system. An AI risk assessment has to also ask what happens when the system behaves exactly as designed and still causes harm — when a model confabulates a citation, when an agent takes an action nobody authorized because a webpage told it to, when a RAG pipeline surfaces a document to someone who was never entitled to see it. None of those are compromises. All of them are risks. This guide is the practical process for assessing them: how to scope, how to score without inventing fake precision, what belongs in the register, how to decide treatment, and where most assessments quietly fail.

Free Tool · Polygraf AI Risk Calculator

Start your assessment with a quantified baseline

Before you build the register, get the number. Polygraf's AI Risk Calculator models your organization's exposure across breach, regulatory, litigation, and reputational risk — and maps which obligations apply to your systems, from the EU AI Act to HIPAA, PCI DSS, and GLBA.

  • Quantified exposure across every major risk category
  • A tailored read on which regulatory regimes govern your AI
  • Gaps surfaced across visibility, data protection, and evidence
  • Modeled reduction from adding inline detection and control
Run the free AI Risk Assessment →
Sample result
Total Potential Exposure
$49.8M
Data breach
Regulatory
Litigation
Reputational

Why AI Risk Assessment Isn't IT Risk Assessment

NIST frames AI risk as fundamentally socio-technical — harms flow not just from code, but from data, deployment context, and human oversight. And it treats AI risk as continuous: systems drift, contexts change, and new risks surface across the lifecycle. Those two properties break several assumptions built into conventional assessments.

Traditional IT risk assessment
Assumes a deterministic system
Asset is a server, app, or database
Threat = an attacker exploiting a flaw
The system behaves the same way each time
Harm requires a compromise
Assess at deployment, review annually
Vulnerabilities have CVEs and patches
Scope ends at the system boundary
AI risk assessment
Assumes a probabilistic, drifting system
Asset is a model, its data, its tools, and its context
Threat includes the system's own normal behavior
Identical input can yield different output
Harm can occur with zero compromise
Assess continuously — drift is a risk event
Failure modes often have no patch, only constraints
Scope includes everything the model can reach or affect
The Unit of Analysis Is the System, Not the Model

The single most common scoping error is assessing the model when the risk lives in the system around it. A well-behaved model wired to an over-permissioned tool, fed by an unfiltered retrieval corpus, with no human checkpoint before it acts, is a high-risk system containing a low-risk model. Assess the whole assembly: the model, its prompts and configuration, its data sources and entitlements, the tools it can invoke, the actions it can take, the humans in the loop, and the population it affects.

The AI risk assessment pipeline — each stage produces an artifact the next one consumes
1 · SCOPE Define the system, context, affected parties 2 · INVENTORY Data, tools, entitlements, dependencies 3 · IDENTIFY Threats, failure modes, harms to people 4 · ANALYZE Likelihood × impact, test on the real system 5 · EVALUATE Compare to risk appetite, prioritize 6 · TREAT Mitigate, transfer, avoid, or accept 7 · MONITOR Continuous evidence, drift, incidents RESIDUAL RISK Must be judged acceptable — Art. 9(5) Monitoring feeds new findings back — the assessment is a loop, not a document Stages 1–2 are the ones teams skip. They're also the ones that determine whether the rest is meaningful.

The Seven Stages, In Practice

1
Scope the system and its context
Write down what the system does, who it affects, what decisions it influences, and what "harm" would mean here. Determine your role — provider or deployer — because obligations differ sharply. Establish risk appetite before you score anything, or you'll rationalize whatever number you land on.
OUTPUT → System description, role determination, risk appetite statement
2
Inventory data, tools, entitlements, and dependencies
Enumerate what data flows in and out, what the model can retrieve, which tools and APIs it can invoke, what permissions it holds, and which third-party models or datasets it depends on. Whatever you fail to list here becomes an unassessed risk by default — and shadow AI means the real list is longer than the sanctioned one.
OUTPUT → Data-flow map, tool/permission inventory, supply-chain list
3
Identify threats, failure modes, and harms
Work from real taxonomies rather than a blank page: OWASP's LLM and Agentic Top 10s for adversarial risk, MITRE ATLAS for attacker techniques, and NIST's Generative AI Profile for the twelve GenAI-specific risk categories. Then add harms to people — the socio-technical half most security teams under-weight.
OUTPUT → Threat list, failure-mode catalog, affected-party harm analysis
4
Analyze — and test on the real system
Estimate likelihood and impact, but ground the estimates in evidence: red-team the actual model with your actual data. A likelihood score derived from a template is a guess wearing a number. Data-extraction, prompt-injection, and agent tool-misuse tests turn assumptions into observations.
OUTPUT → Scored risks, red-team results, evidence pack
5
Evaluate against appetite and prioritize
Compare each scored risk to the appetite you set in stage 1. This is where the heat map earns its keep — not as decoration, but as the forcing function that makes leadership say out loud which risks they're willing to carry.
OUTPUT → Prioritized risk register, escalation list
6
Treat: mitigate, transfer, avoid, or accept
Assign every risk a treatment, an owner, and a date. Note the constraint the EU AI Act imposes here: Article 9 limits the risk management system to risks that can reasonably be mitigated through design, development, or the provision of adequate technical information — and requires that residual risk be judged acceptable.
OUTPUT → Treatment plan, named owners, residual-risk acceptance
7
Monitor continuously and feed findings back
A risk register with no monitoring is not risk management. Instrument the system so that policy decisions, blocked actions, drift signals, and incidents flow back into the register automatically. The assessment is a loop; the document is just its most recent snapshot.
OUTPUT → Live telemetry, updated register, incident linkage

Scoring Without Inventing Precision

Scoring is where assessments go wrong in both directions — either everything is "medium," or a spreadsheet produces a 7.42 that nobody can defend. The workable middle is a coarse likelihood × impact scale, adjusted by a small number of AI-specific amplifiers that genuinely change blast radius. Try it below.

Interactive · Illustrative AI risk scorer

Score one AI risk on one system

Pick a single risk (e.g. "sensitive data leaks into prompts") on a single system.
Likelihood — how plausible is this in the next 12 months?
Rare1
Unlikely2
Possible3
Likely4
Expected5
Impact — if it happens, how bad is the worst credible outcome?
Minimal1
Minor2
Moderate3
Major4
Severe5
AI-specific amplifiers — each meaningfully widens blast radius
The system touches regulated data (PII, PHI, cardholder, MNPI)
+4
The system can take autonomous action without a human checkpoint
+5
There is no reliable audit trail of what it did or why
+4
It affects individuals' rights, access, or livelihood
+5
Select likelihood and impact
Inherent score is likelihood × impact (1–25). Amplifiers add on top, reflecting that identical failures carry very different consequences depending on what the system can reach and whether anyone can see what it did.
Illustrative model for structuring judgment — not a standard, and not a substitute for a formal assessment.
Calibrate the thresholds to your own risk appetite before using it in anger.

Plotting the Register: The Heat Map

The heat map's job isn't to look impressive in a board deck. It's to make prioritization arguments explicit — and to show, at a glance, how many risks sit above the line your organization said it wouldn't cross.

Inherent risk — likelihood × impact (before treatment)
Impact →
5
5
10
15
20
Agent
over-perm.
25
4
4
8
12
Model
drift
16
Prompt
injection
20
PII in
prompts
3
3
6
9
Vendor
risk
12
15
Confab-
ulation
2
2
4
6
8
10
1
1
2
3
4
5
1
2
3
4
5
Likelihood →
Low (1–4) · accept & monitor
Moderate (5–9) · treat on roadmap
High (10–15) · treat this quarter
Critical (16–25) · treat before deploy
Where the Plotted Risks Actually Land

Note where two of the most common risks sit. Prompt injection scores high on likelihood because it needs no malware and no credentials — a sentence in a retrieved document suffices — and high on impact because it inherits whatever permissions the agent holds. Sensitive data in prompts scores even higher on likelihood, because it isn't an attack at all: it's an employee doing their job with the wrong tool. That's the uncomfortable insight of most AI assessments — your top-scoring risks are often the ones with no adversary in them.

Don't Start From a Blank Page: NIST's Twelve GenAI Risks

Stage 3 goes faster when you work from an authoritative taxonomy. NIST's Generative AI Profile (AI 600-1, published July 2024) identifies twelve risk categories unique to or amplified by generative AI, each mapped to suggested actions across the four AI RMF functions. Walk the list against your system; most teams find at least three they hadn't considered.

01
CBRN information
Lowered barriers to chemical, biological, radiological, or nuclear information
02
Confabulation
Confidently generating plausible but factually wrong output
03
Dangerous or violent recommendations
Outputs that encourage harm to self or others
04
Data privacy
Leakage, memorization, or degraded privacy protections
05
Environmental impacts
Resource consumption of training and inference
06
Harmful bias
Discriminatory or homogenizing outcomes at scale
07
Human-AI configuration
Over-reliance, misuse, and poor oversight design
08
Information integrity
Synthetic content degrading the information ecosystem
09
Information security
The AI's own attack surface — prompt injection, model extraction
10
Intellectual property
Training-data IP exposure and output IP risk
11
Obscene or abusive content
Generation of degrading or non-consensual material
12
Value chain & component integration
Risk inherited from third-party models, data, and components

What Belongs in the Risk Register

The register is the assessment's durable output. Anything not in it is not being managed. Every row needs a named owner and a review date — a risk owned by "the AI team" is owned by nobody.

IDRiskInherentTreatmentControlResidualOwner
AIR-01 Employees paste customer PII into an ungoverned AI tool 20 · Critical Mitigate Inline prompt inspection + redaction; enforced AUP; detection logging 4 · Low CISO
AIR-02 Indirect prompt injection redirects the support agent 16 · Critical Mitigate Least-privilege tool scoping; input filtering; human checkpoint on writes 6 · Moderate Head of AI Eng.
AIR-03 Model confabulates a figure in a client-facing summary 15 · High Mitigate Grounded retrieval with citations; human review before external send 6 · Moderate Product Lead
AIR-04 RAG index surfaces documents beyond the user's entitlement 12 · High Mitigate Entitlement-aware retrieval; index ACL review each quarter 4 · Low Data Platform
AIR-05 Third-party model provider changes retention terms 9 · Moderate Transfer Contractual no-training clause; on-prem fallback path; vendor review 6 · Moderate Procurement
AIR-06 Model performance drifts on a minority subgroup 12 · High Mitigate Subgroup performance monitoring with alert thresholds; scheduled re-eval 6 · Moderate ML Lead

Treatment: Four Options, One Honest Answer

🛠
Mitigate
Add controls that reduce likelihood or impact. The default for most AI risks — and the only one that scales.
📄
Transfer
Shift exposure via contract or insurance. Note: you cannot transfer regulatory accountability.
🚫
Avoid
Don't build it, or remove the capability. Legitimate — and underused when the risk can't be constrained.
✍️
Accept
Carry the risk knowingly. Requires a named accountable person signing that it's within appetite.
Residual Risk Is the Part Regulators Read

Article 9(5) of the EU AI Act sets an explicit bar: the residual risk associated with each individual hazard, and the overall residual risk of the high-risk AI system, must be judged to be acceptable. That word "judged" implies a documented judgment by someone with authority to make it. Article 9(3) also scopes the exercise — the risk management system addresses risks that can reasonably be mitigated or eliminated through the system's development and design, or through the provision of adequate technical information.

The practical translation: an assessment that lists inherent risks and stops is incomplete. You need the control, the resulting residual score, and a person who put their name to accepting it. "Accepted" with no signatory is just an unowned risk with better formatting.

Where Assessments Quietly Fail

Five Failure Modes We See Repeatedly
  • Assessing the model instead of the system. The model passes every benchmark; the agent wired around it can still email a customer list. Scope to the assembly.
  • Scoring theater. Numbers derived from templates rather than tests. If nobody red-teamed the actual system with actual data, the likelihood column is fiction.
  • The one-time assessment. AI risk is continuous because models drift and contexts change. An annual PDF cannot capture a system that behaves differently this month than last.
  • A register with no monitoring. If no telemetry flows back into the register, you have documentation, not risk management — and no way to prove a control ever fired.
  • Unowned risks. Every row needs a name, not a team. Accountability that's distributed is accountability that's absent.

"A risk assessment isn't a document you produce — it's a claim you have to be able to defend. Every score should trace to a test, every control to a log line, and every accepted risk to a person who signed for it. Anything else is a spreadsheet with good intentions."

— Polygraf AI, on operationalizing AI risk assessment

One Assessment, Several Frameworks

Done properly, a single assessment produces evidence for multiple regimes at once — which is the entire economic argument for doing it well. NIST's AI RMF is voluntary and offers no certification, but it's the most widely used internal method for generating exactly the evidence that binding regimes demand. ISO/IEC 42001 turns that work into a certifiable management system. ISO/IEC 23894 provides the ISO 31000-aligned risk-management companion. And the EU AI Act binds you to outcomes it doesn't prescribe a method for.

Build the Evidence Once, Map It Everywhere

A risk management policy written once can serve NIST's GOVERN function and EU AI Act Article 9. A red-team and accuracy evaluation report can serve MEASURE, Article 15's accuracy and robustness requirements, and an ISO 42001 audit. The mistake is running three programs; the move is running one and tagging each artifact against every framework it satisfies. Worth noting for planning: NIST states that AI RMF 1.0 is being revised, and in April 2026 it issued a concept note for a forthcoming profile on trustworthy AI in critical infrastructure — so build your mapping to be updatable rather than hard-coded.

Where Polygraf AI Fits in the Assessment Loop

A risk assessment is only as good as the evidence underneath it, and most of that evidence has to be generated at runtime. Polygraf AI's Behavioral Control Plane sits inline at the AI boundary and supplies three of the stages directly. For inventory (stage 2), it discovers shadow AI and surfaces which tools are actually in use and what data reaches them. For treatment (stage 6), it is the control on the highest-scoring rows of most registers — inline detection and redaction of PII, PHI, cardholder data, secrets, and source code before a prompt leaves your environment, which is what moves a risk like AIR-01 from Critical to Low. For monitoring (stage 7), every policy decision — allow, redact, block, which rules matched, who the authenticated actor was — is logged immutably, which is precisely the telemetry that turns a static register into a live one.

Because it runs on-premise with zero data egress and sub-100ms latency, the control layer doesn't add a new risk row of its own. Your residual-risk claim rests on evidence you generated and hold, not on a vendor's assertion that a control was working.

Not legal or compliance advice. This article is a general educational overview prepared by Polygraf AI. The scoring model shown is illustrative and is not a standard, a certification methodology, or a substitute for a formal risk assessment. AI risk obligations are fact-specific and depend on jurisdiction, sector, your role (provider vs. deployer), and system classification. EU AI Act references reflect Regulation (EU) 2024/1689 as amended by the Digital Omnibus on AI; note that obligations for standalone Annex III high-risk systems, including Article 9, now apply from 2 December 2027. Confirm your obligations with qualified counsel and the official texts.
Polygraf AI

Turn Your Risk Register Into Enforced Controls

Polygraf AI enforces policy inline on every prompt and output, and logs every decision immutably — supplying the inventory, the control, and the evidence your assessment depends on. On-premise, sub-100ms, zero data egress.

Request a Demo →
Air-gap ready · NIST AI RMF · EU AI Act · SOC 2
Deploys in under an hour

NEWS & More

Insights & Updates from Polygraf.

Blog Posts

Regulators are asking for AI audit trails enterprises can't produce. Polygraf AI explains what to log and how long to retain it.

To learn more about Polygraf, please get in touch.

At Polygraf, we envision a future where AI augments human capabilities without compromising safety, privacy, or ethical standards. Trust in our commitment to building this future with you.

Products

thank you

Your download will start now.

Thank you!

Please provide information below and
we will send you a link to download the white paper.