AAIR lesson 4: AI trustworthiness and ethics

Trustworthiness is the set of qualities that make it reasonable to rely on an AI system: it performs as intended, treats people fairly, can be understood, protects personal data, and does not cause harm. The NIST AI Risk Management Framework 1.0 (published in 2023) lists seven characteristics: valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. The OECD AI Principles, first adopted in 2019 and updated since, cover similar ground. For a risk professional the skill (one of the outline’s “Other Skills Tested”) is to collaborate with stakeholders so these qualities become managed risks rather than slogans.

From principle to risk statement

An ethical concern becomes manageable once it is written like any other risk: a cause, an event and a consequence, tied to an asset or process. “We value fairness” cannot be tested. “Because the training data under-represents applicants over 60, the screening model may reject qualified older candidates, causing discrimination claims and unfair outcomes” can be rated, owned and treated.

The main concerns, and what each looks like as a risk:

Transparency and explainability are related but not identical. Transparency is about openness: that AI is used, with what data and for what purpose. Explainability is about being able to give a meaningful reason for a particular output.

Proportionality and who decides

Not every use case needs every control. A drafting assistant for internal meeting notes and a model that ranks loan applicants differ greatly in how much harm a failure can do and to whom. The higher the impact on individuals, the stronger the case for fairness testing, explanation, and a human who can override. This is why lesson 1 stressed classifying the use case first. The AI governance and risk topic page covers how that classification feeds policy, and the responsible AI principles page shows a vendor-neutral way of grouping the same concerns.

Ethics decisions are rarely the risk function’s alone. The risk professional’s role is to make the concern visible, rate it using the enterprise method, make sure someone with authority owns the decision, and record any acceptance. Lesson 2 covered those roles.

Worked example: a hiring screen at a 200-person company

A 200-person logistics company buys a hosted tool that ranks job applicants. The vendor reports high overall accuracy. A risk analyst is asked to assess it before rollout.

1. Identify the concerns. Applicant data is personal data, so privacy applies. The ranking affects people’s livelihoods, so fairness and explainability apply. A wrong ranking is also a reliability concern.
2. Write statements. For example: “The tool may rank applicants lower because of attributes correlated with age or gender, leading to discriminatory outcomes and legal and reputational damage.”
3. Rate and own. Impact on individuals is high, and the company cannot see the vendor’s training data, so likelihood is uncertain. The head of HR owns the use case; the risk function owns the rating method.
4. Select controls. Request the vendor’s fairness testing evidence, run a pilot comparing outcomes across groups, add a notice to applicants, keep a recruiter reviewing every rejection recommendation, and agree retention limits for applicant data.
5. Monitor. Track selection rates by group each quarter and set a threshold that triggers review.

Overall accuracy was never the answer. A model can be accurate on average and still unfair to a subgroup, which is why the control is group-level testing.

Easy to confuse

• Accuracy vs fairness. High accuracy does not show equal treatment across groups.
• Explainability vs transparency. Telling users that AI is involved is transparency; giving the reason for one decision is explainability.
• Privacy vs security. Security protects data from unauthorised access; privacy also asks whether the data should be used at all for this purpose.
• Ethics vs compliance. Lawful use can still breach the organisation’s stated values, and the reverse is also possible. Both need assessing.
• Removing a protected attribute vs removing bias. Other fields can act as proxies for it.

Key points

• Trustworthiness is a set of characteristics; frameworks such as NIST AI RMF and the OECD principles name them, and a risk professional uses them as a checklist of concerns.
• Convert each concern into a risk statement with cause, event and consequence, so it can be rated, owned and treated.
• Fairness needs group-level testing and monitoring, not just overall accuracy.
• Transparency is openness about AI use; explainability is a reason for a specific output.
• Controls scale with the impact on people, and human oversight matters most where decisions affect individuals.
• The risk function makes ethical risk visible and recorded; accountable business owners make the decision.

Check yourself

This lesson covers subtopic 1F, the last of the six areas in Domain 1 (AI Risk Governance and Framework Integration, 37%); Domain 2 begins with lesson 5. The governance framework recap summarises the whole domain.