AAIA lesson 5: Data management for AI
Data management for AI is the discipline of controlling where training and operating data comes from, how it is labelled and prepared, how it is split for training and testing, and how long it is kept. It matters more than almost any other area because a model’s behaviour is a product of its data: errors, bias and unlawful sourcing do not stay in the dataset, they are carried into every output. This lesson covers subtopic 2A, Data Management specific to AI, the start of Domain 2; a supporting task asks the auditor to evaluate data input requirements such as appropriateness, bias and privacy.
Following the data
The figure shows the path: data sources, collection and labelling, preparation and splitting, training, model, outputs. The checks under each stage map to questions an auditor can put to any pipeline; the paragraphs below also cover bias, access and retention, which run across all stages.
Provenance and lineage. Provenance is where data originated and under what terms; lineage is the record of every transformation since. Together they let an organisation show that data was lawfully obtained and suited to the purpose. Evidence: source agreements or licences, a data catalogue, and lineage records that connect a trained model version to specific dataset versions.
Quality. Does the data represent the situation the model will face? The relevant dimensions are accuracy, completeness, consistency, timeliness and representativeness. Evidence: profiling results, defined acceptance criteria, treatment of missing values and duplicates, and records of what was excluded and why.
Labelling. Labels are human judgements, and a supervised model learns them as truth. Evidence: labelling instructions, labeller training, checks on agreement between labellers, and review of disputed cases. Outsourced labelling adds a third party and a data-handling question.
Bias. Data can under-represent groups, reflect past discrimination or contain proxies for protected attributes. Evidence: representation analysis by relevant group, and outcome testing later in the lifecycle.
Access. Anyone able to alter training data can alter the model. Evidence: access lists, dataset change logs and integrity checks.
Retention. Keep data no longer than the purpose requires, and know what exists in copies, extracts and logs. Evidence: a retention schedule applied to training sets, prompts and outputs, and disposal records.
Splitting data: why three sets
A dataset is normally divided into a training set that the model learns from, a validation set used to tune choices during development, and a test set held back for a final, unbiased performance estimate. The overfitting and metrics page shows the mechanics.
The control point is separation. If test records leak into training, performance figures look better than reality, because the model has effectively seen the answers. Leakage can be subtle: duplicates across sets, a split by record when it should be by customer, a random split of time-ordered data (the model then trains on the future of what it is tested on), a feature that already contains the answer (for example a field filled in only after a claim is confirmed fraudulent), or tuning repeatedly against the test set. An auditor asks how the split was made, whether it was documented and reproducible, and whether the test set was used only once at the end.
Worked example: a churn model at a telecom
A telecom with 3,000 staff (a different organisation) builds a model that predicts which customers will cancel. Reported accuracy is excellent. The auditor works through the pipeline.
- Provenance. Part of the data comes from a marketing partner. The agreement covers analytics but not model training, so lawfulness of use is uncertain.
- Quality. Records from one region are missing for several months after a system migration. The model has little to learn from for that region, so its accuracy there is untested.
- Labelling. “Churned” was defined as no activity for 30 days, which also catches seasonal customers. The label encodes a choice nobody reviewed.
- Splitting. Each customer appears several times with monthly snapshots, and rows were split at random. The same customer is in both training and test, which inflates the accuracy.
- Retention. Snapshots are kept indefinitely in a team share.
The overall finding is that the headline accuracy cannot be relied on, and the reasons are all data management failures. The recommendation would be to confirm the partner terms, rebuild the split by customer, document the label definition, quantify performance by region and put the dataset under a retention schedule.
Easy to confuse
- Provenance versus lineage. Provenance is origin; lineage is the journey.
- Validation set versus test set. The first guides development; the second is the final independent check.
How a practitioner applies it
Ask for the dataset register and pick one model in production. Trace its versions back to datasets and sources, and ask for the approval that allowed each source to be used. Re-perform something simple: check that the split has no overlap, or recompute a quality metric from the raw data. Compare the documented label definition with a sample of labelled records. See the data classification page for how sensitivity affects handling.
Key points
- A model inherits the strengths and flaws of its data, so data is the primary audit target in AI operations.
- Provenance and lineage should link a model version to dataset versions and source terms.
- Labels are decisions and need instructions, training and review.
- Training, validation and test data must stay separate; leakage inflates reported performance.
- Retention applies to training sets, prompts, outputs and logs.
Check yourself
Question 1. An auditor finds that records for the same customer appear in both the training and the test sets of a production model. What is the MOST likely consequence?
- A. Inference will be slower in production
- B. The training set will be too small to be representative
- C. Reported performance will be overstated
- D. Outcomes will be systematically less fair to minority groups
Show answer
Answer: C
When the model has already seen a customer’s records in training, testing on the same customer overstates how well it will generalise, so reported performance is inflated. Leakage does not by itself slow inference or shrink the training set, and it says nothing about the model’s fairness across groups.
Want more questions like this? Full AAIA practice tests →
Question 2. Which evidence BEST supports management's claim that a model's training data was lawfully obtained?
- A. Source agreements covering training use, linked by lineage to the model version
- B. A statement of how many records the dataset contains
- C. Accuracy results from the validation set
- D. An inventory of the storage devices holding the data
Show answer
Answer: A
Source agreements and licences that cover the training use, linked through lineage to the model version, directly address lawful sourcing. A data volume figure, accuracy results or a storage inventory say nothing about the terms of use.
Want more questions like this? Full AAIA practice tests →
Question 3. A supervised model's labels were produced by an outsourced team without written instructions. What should the auditor be MOST concerned about?
- A. The cost of the outsourcing contract
- B. The size of the labelling team
- C. The speed at which labels were delivered
- D. Inconsistent labelling that the model will learn as truth
Show answer
Answer: D
Labels are treated as ground truth during training, so inconsistent or undocumented labelling embeds errors and judgements into the model. Cost, team size and turnaround are operational matters and do not bear on what the model learns.
Want more questions like this? Full AAIA practice tests →
The exam-focused summary is on the AI operations recap.