AAIR lesson 11: third-party AI risk

Most organisations never build a model from scratch: they call a provider’s API, download a pretrained model, buy a dataset or use a vendor product that quietly contains AI. Each is a third-party dependency, and the buyer stays accountable for outcomes while seeing little of what is inside. AI supply chain risk management (subtopic 3E in the ISACA AAIR outline) means finding those dependencies, judging them, binding them with contracts and watching them over time. Two items under “Other Skills Tested” in the outline point at it: advising on AI risk in contracts and service agreements, including data usage and intellectual property, and evaluating AI risk within supply chain risk management.

What counts as the AI supply chain

Classic vendor risk looks at a supplier’s company: financial health, security posture, location. An AI supply chain adds layers that sit beneath the supplier’s name.

Ask “who is behind the vendor?”: the vendor may rely on a fourth party’s model, so the chain is longer than the contract suggests. See AI governance and risk for the wider programme.

Due diligence before and after signing

Due diligence is proportionate: the more critical the process and the more sensitive the data, the deeper the questions. Typical themes:

• Provenance and integrity. Where did the model and data come from, and can the artefact be verified?
• Data handling and rights. Are prompts stored, are they used to train the provider’s models, and was its training data lawfully obtained?
• Change management. How is the buyer told about model updates, retirements or sub-processor changes?
• Assurance. Is there independent evidence, and what exactly does its scope cover?

Diligence continues after signature, because a hosted model can change under the buyer’s feet. Ongoing monitoring (checks on a fixed test set, review of provider notices, periodic reassessment) belongs to the same control.

Contract and service-agreement clauses

The contract turns findings into enforceable terms. Legal counsel drafts; the risk professional advises on what the assessment requires. Typical topics:

• Data usage: whether inputs and outputs may be used to train or improve the provider’s models, retention periods, deletion on exit, location of processing, and sub-processors.
• Intellectual property: ownership of outputs, treatment of fine-tuned models, and who bears the risk if output infringes others’ rights (an indemnity).
• Change notification: advance notice of material model changes, with time to test.
• Service levels, audit rights, incident notification times, and exit terms covering data return and transition help.

Enforceability depends on jurisdiction, so get current legal advice rather than assuming a clause works.

Worked example: the support chatbot

A 200-person software company wants a customer-support assistant that calls a hosted language model, retrieves answers from its knowledge base and stores chat logs with a second vendor. The risk professional maps three third parties: the model provider, the knowledge-base platform and the logging vendor.

Findings: the default terms let the provider retain inputs for model improvement; there is no notice period for model updates; and switching providers would take months because prompts and tests are written for one model. Recommended actions: negotiate a no-training clause and deletion on exit, require change notification with a test window, keep a fixed evaluation set to detect behaviour shifts, and document a fallback such as a human-handled queue. Appetite allows moderate dependency for a non-critical channel, so the remaining risk is accepted with an owner and a review date.

Concentration and dependency

Concentration means many processes depending on one provider, model family or region. Switching AI providers is costly because prompts, tuning and tests are model-specific, so an outage or withdrawal hits every dependent process at once. Mitigations: multi-provider designs for critical uses, tested exit plans, and an inventory showing which process relies on which model. This leads into continuity planning in the next lesson.

Easy to confuse

• Certificate versus fitness: a certificate shows a management system exists within a stated scope, not that the model suits your use case.
• Transferring loss versus accountability: an indemnity moves some financial loss; the buyer still answers for decisions.

Key points

• The AI supply chain spans hosted models, pretrained weights, datasets, embedded AI features and infrastructure.
• Due diligence is proportionate and continues after signature, because providers change models.
• Contracts should settle data usage, IP, change notification, audit rights and exit.
• Concentration on one provider is a risk in itself: keep an inventory and a tested fallback.

Check yourself

This lesson covers subtopic 3E in Domain 3, AI Risk Program Management (42% of the exam); the programme management recap puts it alongside the other five areas of that domain.