AAISM lesson 10: Securing the AI life cycle

Securing the AI life cycle means attaching a specific security check to each stage, from choosing a model to maintaining it in production, so that no stage passes risk silently to the next. The outline’s subtopic 3B names model selection, training and validation as examples, and a supporting task asks the security manager to advise on security risk and controls across the solution development life cycle. Domain 3 carries 38% of the exam.

Five stages, five gates

Select: vetting and provenance. The first decision is build, buy or adapt. Using a hosted API moves the model’s protection to the provider; using open weights means the organisation inherits whatever is inside them and must host them itself. Provenance is a verifiable record of where a model or dataset came from. Practical checks include obtaining artefacts only from sources the organisation has approved, verifying published hashes or signatures, and preferring serialisation formats that cannot execute code when loaded. Unverified downloaded model files are a known route for malicious code, so a model is handled like third-party software.

Train or fine-tune: data integrity. Training data is verified for source, protected from modification and screened for sensitive content that should not be learned. Control who may start a training run and from where, because an unauthorised fine-tune changes production behaviour. Training environments are separated from production and logged. See fine-tuning, RAG and prompting for the options being secured.

Validate: security testing. Before release the system is tested against defined criteria, not general impressions. Testing covers accuracy against the intended use, robustness to adversarial and malformed input, resistance to prompt injection, leakage of sensitive data, fairness across groups where decisions affect people, and safety of generated content. The evaluation approach is explained in LLM evaluation. Results are kept as evidence, since they are what a regulator or auditor will ask to see.

Deploy: access and configuration review. Release checks confirm that credentials are held in a secrets store, the gateway and guardrails are in place, tool permissions are minimal, logging is on, and rollback is possible. Deployment of models follows the change management used for other production systems.

Maintain: monitoring and patching. Models drift as the world changes, libraries get vulnerabilities, and providers update models. Maintenance includes monitoring quality and abuse signals, patching the surrounding stack, retraining or replacing models on defined triggers, and retiring models that are no longer supported. MLOps practice (see MLOps basics) makes these steps repeatable.

Release criteria as a management tool

A gate only works if the pass condition is written down in advance. For a customer-facing assistant, criteria might include a maximum rate of successful injection attempts in the red-team set, no leakage of seeded test secrets, accuracy at or above an agreed level on a fixed evaluation set, and sign-off by the named risk owner. The security manager does not set every number alone; business and risk owners agree thresholds, and the manager ensures that they exist, are testable and are applied.

Worked example

A 400-person insurer plans to fine-tune an open-weights model on past claims notes. At selection, the team downloads the weights from a community site; the security manager requires the artefact to come from the approved repository, with its hash verified and the licence reviewed. For training, notes are stripped of direct identifiers, stored in an isolated environment and fine-tuning is run by a service account that only the ML platform team can use. In validation, tests include attempts to extract names from the model and a fairness comparison across customer segments; one test fails on leakage, so release is blocked and the data is cleaned and retrained. At deployment the model sits behind the standard gateway. Monthly evaluations and a retraining trigger on measured drift cover maintenance.

Easy to confuse

• Testing vs validation. Security testing probes for weaknesses; validation confirms that the system meets its intended purpose and criteria. Both are needed.
• Provenance vs integrity. Provenance records origin; integrity shows that the artefact is unchanged since then.
• Retraining vs patching. Patching fixes the software around the model; retraining changes the model’s behaviour and goes back through validation.

Key points

• Each life cycle stage has one security gate, written down with a pass condition.
• Third-party model files are treated as untrusted software until vetted.
• Controls on who can train and fine-tune protect production behaviour.
• Validation evidence is kept for auditors and regulators.
• Maintenance needs monitoring, patching and defined retirement.

Check yourself

This lesson covers subtopic 3B of Domain 3, AI Technologies and Controls (38%); the recap is at technologies and controls explained.