Dataset quality
AI dataset quality: checks before licensing
A dataset quality review should answer a specific question: are these records usable for the proposed task? Begin with completeness, duplication, definitions, coverage and known limitations, not a single impressive-looking quality score.
Last checked
Record what was checked, on which version and for which purpose. A clean file is not proof of useful examples, permission to share or buyer demand.
Define the task before judging quality
The W3C Data Quality Vocabulary provides a way to describe quality so a user can judge fitness for purpose. It does not supply one complete definition of quality for every dataset. The same caution is useful in a commercial AI discussion.
Write the intended task in plain language. Is someone interested in how a problem was diagnosed, whether a response was correct or how a process changed over time? Ask what evidence would be needed for that task. Without a defined purpose, a clean-up can consume time while leaving the important question unanswered.
This is a suggested internal review method, not a buyer acceptance standard or a Fairlode certification. Keep the checks within systems and permissions your business already controls. Do not upload records to an external AI tool to obtain a quick score.
Count usable examples, not just rows
First define the unit being counted. A conversation may contain several messages; an issue may have several updates; a repair may have several visits. Describe the relationship before presenting a total. Keep record counts separate from any count of complete examples.
Choose the fields needed for the proposed task and note where they are absent. Do not automatically discard every incomplete record. Explain which question each omission prevents you answering. A missing completion date and a missing outcome description are not necessarily the same limitation.
An illustrative support archive might have a closure flag but no evidence that a suggested fix worked. Report that gap directly. Do not turn "closed" into "successfully resolved" to make the collection look more attractive. Use a data dictionary to record the actual meaning of the field.
Distinguish duplicate records from repeated events
Ask why apparently repeated entries exist. They might be copied exports, repeated boilerplate, successive versions or genuinely separate events. Decide what counts as a duplicate for the proposed task before removing anything.
Preserve the original source and keep a record of any transformation in an authorised working environment. A cleaned copy should not become the only explanation of how the records were selected. If the review uses a sample, state how it was selected and avoid presenting its findings as a full-population result.
For example, an invented repair log might repeat a standard safety note in every entry while recording a different fault each time. Removing all similar-looking records would be a questionable shortcut. Ask whether the repetition obscures useful differences or is simply part of the reporting process.
Check who supplied the labels and outcomes
List the labels a reviewer might rely on: issue category, outcome, severity, success or failure. Ask who assigned them, what instructions they followed and whether those instructions changed. Separate a person's judgement from a measured event and a system-generated default.
Where the task depends on correctness, ask how the claimed answer was checked. If no independent check exists, describe that limitation. Do not retrospectively invent validation because the records have been used operationally for years.
An illustrative distinction is a technician selecting "repair complete" versus a later test confirming that equipment passed a defined check. Either might be useful for a particular purpose, but they describe different evidence. A buyer should not have to discover that distinction after preparation work is finished.
Look for gaps in time and coverage
Organise the review around the scope you intend to describe. Which periods, products, languages or workflows are included? Which are missing or deliberately excluded? Note changes in collection practice, system migrations and category definitions.
Do not imply that a collection represents an entire industry because it covers a long period in one business. State the boundary you can support. An archive from one product line may document that product line well without answering questions about others.
Ask whether selection decisions have favoured only straightforward cases or successful outcomes. Keep the selection explanation with the review. These are prompts to investigate your own collection, not claims about the prevalence of any problem in commercial datasets.
Finish with a decision, not an unexplained score
Summarise the review in a short decision note: the intended task, version checked, method, findings, limits and next step. Use "not checked" where appropriate. If you use percentages internally, include the underlying counts and method; do not substitute a percentage for an explanation.
Separate work that is essential for the proposed task from improvements that would merely make the archive tidier. Ask who would pay for further preparation and what evidence of buyer interest justifies it. It may be sensible to stop with a documented limitation rather than begin an expensive repair project.
Quality findings do not answer the privacy and rights questions. Read the anonymisation comparison and the readiness checklist for those boundaries. For a Fairlode enquiry, describe the known limitations without attaching the records or claiming that the collection is approved for AI use.
General information, not legal advice, a valuation or a buyer offer. Illustrative examples are not customer case studies.
Sources and further reading
Sources checked 16 October 2026. Historical research is background, not evidence of current buyer demand.