The InfoSecured guide
AI assurance: from claims to confidence.
AI assurance is the process of evaluating and communicating evidence about an AI system’s trustworthiness. It helps people judge whether a system is suitable for a particular use, what its limitations are, and which risks need attention.
Start with a claim. Examine what supports it. Make the limits of your conclusion clear.
The assurance question
Can we rely on this AI system
for this use?
-
A specific claimWhat needs to be true?
-
Relevant evidenceWhat has actually been examined?
-
A bounded conclusionWhat can we reasonably conclude?
01 / Foundations
Where does assurance fit in AI governance?
When an AI system influences a decision, different people need different answers. A business owner needs to know whether to use it. A reviewer needs to understand its limits. Someone affected by the decision may need a way to question it. Assurance makes the basis for those decisions open to examination.
Governance, risk management, and assurance have related but distinct jobs:
- AI governance
- Who decides, and under which rules?Sets accountability, decision rights, policies, and oversight.
- AI risk management
- What could go wrong, and how will we respond?Identifies, evaluates, treats, and monitors risks in context.
- AI assurance
- What supports our confidence in this system?Examines and communicates evidence against defined claims or criteria.
This distinction draws on DSIT’s assurance guidance and the NIST AI Risk Management Framework. A specific engagement may use different terminology or review criteria.
InfoSecured focuses on the connections: how a requirement becomes a safeguard, how that safeguard is examined, and how the findings change a decision. A documented policy can establish an expectation. Testing, observation, and operating records help establish what happens in practice.
02 / Worked example
A convincing answer can still cross a boundary.
Imagine a support assistant that searches internal documents and drafts answers for employees. Its answers may look accurate while including information a particular employee should never be able to retrieve.
The claim to examine
“The assistant only uses documents the requesting employee is authorized to access.”
Identify the assistant version, document collections, access roles, and point in time covered by the review. Include retrieval and the generated answer in the test boundary.
Compare configured permissions with retrieval traces and answer outputs for authorized and unauthorized users. Include changed permissions and attempts to obtain restricted information.
Suppose a test shows that a cached answer still exposes restricted text after a user’s access has been revoked. The review has found a route the original claim does not account for.
The claim is not supported for the tested configuration. Restrict the affected use, assign a correction, and retest the access-revocation path before reconsidering that decision.
An InfoSecured teaching example, not a client case or a reported test result. Actual test design depends on the system architecture, threat model, and intended use.
The useful output is a decision with a reason: what was tested, what failed, who owns the response, and what evidence would justify a different conclusion. A collection of documents becomes valuable when a reviewer can follow that reasoning.
03 / Evidence
What counts as AI assurance evidence?
AI assurance evidence is information used to evaluate a claim about an AI system. It may include test results, system records, observations, review decisions, and source documentation. Its value depends on how directly it addresses the question being examined.
Use the following as a starting point when planning a review. Select records because they help answer a question, then check their quality and relevance.
| Review question | Evidence to examine | What to challenge |
|---|---|---|
| What system are we evaluating? | System inventory, intended use, version, data sources, dependencies, and owner. | Do these records describe the deployed configuration and the people it affects? |
| Does it perform acceptably? | Evaluation method, test cases, results, thresholds, and failure analysis. | Do tests reflect the intended conditions and relevant groups, including difficult cases? |
| Do safeguards operate? | Control tests, access reviews, operational logs, exception records, and corrective actions. | Is there evidence of operation over the review period, beyond the written policy? |
| Can people intervene? | Reviewer permissions, available information, intervention tests, escalation records, and decision rationale. | Can the responsible person recognize a problem and act before the consequence? |
| What depends on a vendor? | Version-specific documentation, testing summaries, change notices, and agreed responsibilities. | What remains unexamined, and what must the customer test in its own environment? |
| Does the conclusion still hold? | Monitoring results, incidents, configuration changes, model updates, and review triggers. | What changed after the evidence was collected? |
A useful evidence check
Record the source, collection date, relevant system version, review scope, method, and result. Ask whether someone else can examine the record and understand why it supports—or weakens—the claim.
For more focused questions, explore human oversight, vendor AI risk, and model risk review.
04 / The review
How do you put AI assurance into practice?
Start with a decision that needs support. The sequence below is InfoSecured’s practical starting point for organizing the work; adapt the depth and methods to the system’s risks and the review criteria.
-
Define the decision and the claim.
State the intended use, affected people, system boundary, and criteria. Make the claim specific enough that contradictory evidence could change your view.
-
Choose how you will examine it.
Select suitable tests, observations, interviews, or document reviews. Identify the competence and independence needed for the work, and record gaps in access.
-
Evaluate what the evidence supports.
Compare results with the criteria. Investigate contradictions, missing records, uncertain results, and conditions that were not tested.
-
Record a conclusion someone can act on.
Explain the findings and their limits. Identify any conditions of use, corrective actions, responsible owners, and the decision-maker accepting the remaining risk.
-
Set the triggers for another review.
Decide which model, data, configuration, or use changes require reassessment. Include incidents and unexpected outcomes as well as scheduled checks.
NIST AI RMF 1.0 organizes risk management around Govern, Map, Measure, and Manage. It offers a reference for lifecycle risk work; the sequence above is not an official NIST audit procedure. Explore the framework.
05 / Questions that matter
Understand what the conclusion means.
Is AI assurance the same as certification?
No. Assurance covers ways of evaluating and communicating evidence. Certification is a particular assessment route against specified requirements. A review or completed checklist should not be presented as certification.
How much evidence is enough?
There is no universal document count. Define the claim and criteria first, then assess whether the evidence is relevant and sufficient for the decision, considering possible consequences and remaining uncertainty. A serious unresolved contradiction deserves attention even when the file is otherwise complete.
What does “audit-ready” mean?
Audit-ready records are organized so a reviewer can trace scope, criteria, tests, decisions, and supporting evidence. Preparation makes examination possible; the reviewer still has to evaluate the substance and reach a conclusion.
Does assurance establish compliance?
An assurance review supports conclusions within its defined scope and criteria. Compliance with particular requirements needs an assessment against those requirements. A broad claim that an AI system is “compliant” should identify exactly what was assessed.
Put it into practice
Give your next review a clear structure.
The AI Assurance Evidence Review Kit connects claims, risks, controls, evidence, tests, human oversight, findings and decisions across fourteen Excel worksheets.
Version 2.2.1 includes Start Here, Field Guide and a connected Worked Example. Use the workbook to define the review, assess supporting records, assign follow-up and record the overall decision with its authorized scope and reassessment triggers.
Have a question about the work or a potential collaboration? Contact InfoSecured.
06 / Sources & approach
The basis for this guide
This guide combines published assurance and risk-management guidance with InfoSecured’s practical interpretation. The knowledge-assistant example, evidence-selection table, and review sequence are illustrative teaching material. They do not represent a completed client engagement or an official assessment standard.
- DSIT — Introduction to AI assuranceFoundations of assurance, its relationship to governance, and the range of assurance techniques.
- NIST — AI Risk Management FrameworkRisk management across the AI lifecycle. This guide references AI RMF 1.0; consult NIST for framework updates.
- NIST AI Resource Center — AI RMFThe Govern, Map, Measure, and Manage functions and supporting framework material.
Published by InfoSecured. Read our editorial standards or suggest a correction.