AAISM lesson 6: AI risk assessment

AI risk assessment identifies what could go wrong with an AI use, estimates how likely and how damaging it is, compares the result with the organisation’s threshold and decides what to do. Subtopic 2A opens Domain 2 (31%), and the supporting tasks add impact assessments, regulatory conformity and monitoring for reasons to reassess. Two facts matter: the method is the one managers already know, and AI adds risks that move over time even when nobody changes the system.

Identifying and assessing the risk

Start from the use case, not the technology. A model that drafts internal meeting notes and one that approves loans use similar technology and carry very different risk. The NIST AI RMF expresses the same order in its Map function: establish purpose, context, users, affected parties and known limits before measuring anything.

Risk sources specific to AI include biased or low-quality data, inaccurate or invented outputs, data leakage, manipulation by adversaries, over-reliance by users, opaque decisions, third-party dependencies and drift. Each is described as a scenario (cause, event, consequence) and rated for likelihood and impact. The qualitative versus quantitative topic explains the rating approaches; AI work often uses qualitative scales supported by test measurements, because loss data is scarce.

An AI impact assessment widens the lens beyond the organisation to the people affected: individuals, customers, groups who might be treated unfairly, and society. It is performed before deployment and is proportionate to the use, with heavier effort where rights, safety or money are at stake. It also supports regulatory conformity, since regimes such as the EU AI Act attach assessment, documentation and oversight duties to higher-risk uses.

Thresholds and appetite

Risk appetite is the amount of risk the organisation is willing to pursue; a threshold or tolerance turns that into a limit for a specific measure. Because AI performance is probabilistic, thresholds are best expressed as numbers: a maximum error rate, a minimum share of answers grounded in sources, a ceiling on difference in outcomes between groups, or a requirement that a human reviews decisions above a value. If a measure breaches its threshold, that triggers a defined action rather than a debate.

Treatment and residual risk

The figure shows the decision. Risk within the threshold can be accepted and monitored. Risk above it is treated by mitigating, transferring or avoiding it; the table also lists acceptance, which is the route for risk already within the threshold.

After treatment comes residual risk, the risk that remains. If it is still above the threshold, further treatment or escalation is needed. Acceptance of residual risk is a decision for the business owner, documented with a rationale and a review date, not a default.

Reassessment triggers

AI risk is not static, so assessments are scheduled and event-driven. Typical triggers are a change of model or version (including a silent vendor update), new data sources, a change of purpose or user group, an incident or near miss, performance drift or monitoring alerts, new threats or vulnerabilities published, and new regulation or guidance. Monitoring of internal and external factors, as the supporting task describes, exists to notice these.

A worked example

A 400-person insurer proposes a model that flags suspicious claims for investigators. The mapping step notes that flagged customers may face delay or denial, so affected parties are policyholders. The impact assessment finds possible unfair flagging of certain groups. Appetite for unfair outcomes is low, so the threshold requires the flag rate difference between groups to stay under a stated limit and every flag to be reviewed by a person before any action. Testing meets the first condition only after retraining. Residual risk is within threshold, the claims director accepts it and sets quarterly review plus reassessment on any model change. Without these steps the decision would depend on enthusiasm for the tool.

Easy to confuse

• Risk appetite versus tolerance. Appetite is broad; tolerance is a specific limit.
• Inherent versus residual risk. Inherent is before controls; residual is after.
• Transfer versus avoid. Transfer moves the financial effect; it does not remove responsibility.

Key points

• Start from the use case and the people affected.
• Impact assessment precedes deployment and scales with the stakes.
• Express thresholds as measurable limits.
• Accepting a documented, in-threshold risk is a valid treatment.
• Residual risk needs an owner, and triggers drive reassessment.

Check yourself

This lesson opens Domain 2 (31%) with subtopic 2A; the risk management recap summarises the domain for revision.