AAIR lesson 10: AI risk metrics, reporting
AI risk metrics turn a risk that is hard to see into numbers that can be watched against agreed limits. A key risk indicator (KRI) is a measure that signals a change in exposure before a loss happens, and it only works if it has a threshold, an owner and a defined response. AI makes this more urgent than with conventional systems, because a model can lose accuracy or start treating groups unequally without any change being made to it.
From risk to indicator
Start with the risk and ask which observable measure would move first if it worsened. The measure should be quantifiable, available on a regular schedule, and tied to a decision. Leading indicators (a rising share of low-confidence answers) warn of trouble; lagging ones (complaints upheld) confirm it. A mix is best.
| AI risk | Possible indicator | Threshold idea |
|---|---|---|
| Declining accuracy (drift) | Error rate on a rolling labelled sample | Escalate if above an agreed percentage for two periods |
| Unfair outcomes | Gap in approval rates between groups | Investigate if the gap exceeds an agreed limit |
| Data leakage | Count of outputs flagged by a sensitive-data scanner | Any confirmed case triggers review |
| Human oversight weakness | Share of reviewed outputs overridden or approved without change | Near-zero overrides prompts a check of the review step |
Thresholds should link back to risk tolerance from the previous lesson. A common set-up has an amber level (early warning, investigate) and a red level (tolerance breached, act). Vague statements such as “the model is reliable” cannot be monitored; “error rate below 5% on the weekly sample” can.
Continuous monitoring
An annual review fits a system that changes only when someone changes it. AI systems can change through their inputs. Customer behaviour, language and data sources shift, so a model that was validated at launch may be less accurate months later. Monitoring therefore needs to run continuously or at short, fixed intervals, with automated checks where possible and human sampling where judgement is needed. It should also be triggered by events: a new model version, a provider update, a change in the data feed or a new use of the system.
An alert nobody receives is a control in name only: each indicator needs a named owner and an agreed response, such as pausing the feature or sending outputs to manual review.
Reporting to three audiences
The figure below shows metrics and KRIs feeding monitoring, which feeds reports for operations, management and the board.
| Audience | Needs | Typical content |
|---|---|---|
| Operations | Detail to act today | Live indicator values, alerts, open issues, test results |
| Management | Trends and decisions | KRIs against thresholds, risk register movement, remediation progress, upcoming changes |
| Board | Assurance and direction | A small set of headline indicators, top AI risks against appetite, significant incidents, regulatory developments, decisions requested |
The same facts are shown at different levels of abstraction. The board does not need error rates by model; it needs to know whether the organisation is inside its appetite, what has changed, and what it is being asked to decide. Reports should show trends, state data limits and sit within the enterprise risk report, not a separate silo.
Worked example: a 1,000-person retailer
A 1,000-person retailer uses an AI model to forecast demand and another to answer customer chats. The risk team defines five indicators: forecast error against actual sales, share of chat answers rated unhelpful, number of chats where personal data was exposed, count of unapproved AI tools found on the network, and provider model changes without notice. Each has amber and red thresholds and an owner.
In month four, forecast error crosses amber after a new product range launches. Operations receives the alert, finds that the model has little data on the new range, and adds a manual adjustment. Management’s monthly report shows the trend and the remediation. The board report that quarter carries two lines: AI forecast risk is back within tolerance, and one chat privacy incident was contained. Because the indicators had agreed owners and responses, the issue was handled before it affected stock levels in a visible way.
Easy to confuse
- KRI vs KPI: a KPI measures performance against a goal; a KRI measures exposure to a risk. A single measure can serve as both, but the purpose differs.
- Leading vs lagging: early signal versus confirmation after the event.
- Monitoring vs testing: monitoring watches continuously; testing checks at a point in time.
Key points
- A KRI needs a measurable threshold, an owner and an agreed response.
- Use leading and lagging indicators, and link thresholds to risk tolerance.
- AI needs continuous or short-interval monitoring because behaviour can change without any system change.
- Tailor reports: detail for operations, trends for management, appetite and decisions for the board.
- Test that alerts reach people who can act.
Check yourself
Question 1. A dashboard shows that a hiring-support model is 'performing well'. Which change would MOST improve it as a risk indicator?
- A. Show the word ‘well’ in green
- B. Add more charts to the page
- C. Extend the history to five years
- D. Replace it with a measured value, an amber and red threshold, and a named owner
Show answer
Answer: D
An indicator needs a measurable value, a threshold and an owner so a breach triggers action. A word like ‘well’ cannot be breached, and adding colours, charts or a longer history does not create a threshold.
Want more questions like this? Full AAIR practice tests →
Question 2. A model passed validation at launch. Customer behaviour is changing quickly. Which monitoring approach is MOST appropriate?
- A. Continuous or frequent checks plus review on events such as new versions or data changes
- B. One review each year
- C. Waiting for users to complain
- D. No monitoring, since validation passed
Show answer
Answer: A
Because inputs change, performance can decline between reviews, so frequent or continuous checks with event triggers detect problems sooner. A single annual review is too infrequent, relying on user complaints is lagging and incomplete, and stopping monitoring after validation ignores drift.
Want more questions like this? Full AAIR practice tests →
Question 3. The risk practitioner prepares a quarterly report for the board on AI. Which content is MOST suitable?
- A. Raw system logs for every model
- B. Headline indicators against appetite, significant incidents and decisions requested
- C. A table of error rates per model version
- D. A list of all configuration settings
Show answer
Answer: B
The board needs headline indicators against appetite, significant incidents and decisions required. Raw logs and model-level error tables are operational detail, and a list of technical settings does not show whether risk is within appetite.
Want more questions like this? Full AAIR practice tests →
This lesson covers subtopic 3D of Domain 3, AI Risk Program Management (42%), and the recap page AI risk programme management sets it beside the other five areas of the domain.