Security review

The questions your security review will ask, and the answers that pass.

24 real reviewer questions across InfoSec, Privacy & Legal, Risk & Compliance, and Procurement. Each one shows what passes, the control that produces it, and the evidence you hand over. Filter to your situation. Copy it as Markdown. Take it into the room.

Updated October 26, 2026 First published July 31, 2026

23 min read

Markdown

Most AI projects in mid-market and enterprise firms stall after the model works. They stall in the review that comes next: InfoSec, privacy and legal, risk and model governance, procurement. There are four gates, four owners, and four different questions. Usually no shared checklist says what a passing answer contains.

This is that checklist. It goes past "have a policy". For each item you get the reviewer's question and the reason they ask it. You also get what a passing answer contains, the engineering control that produces it, and the evidence you hand over.

Passing this review ships one feature. The AI governance layer is the standing regime that keeps the answer true a year later. It holds the register, the oversight checkpoint, and the records that this checklist assumes already exist.

The four gates

Four gates, four different owners.#

Each gate tests something different. Know which gate is asking before you answer.

Four review gates run in order. A rejected gate sends the review back to the start.

InfoSec — AppSec & Security Engineering

Owner: AppSec or security engineering, whoever owns your SDLC security gate

What an attacker can reach, and what happens when someone abuses it. This is the blast radius of a compromised prompt, key, or session.

No tool allowlist. The agent can call anything that a session with its credentials could call.
No answer for what a compromised or malicious prompt could make the agent do downstream
Secrets or credentials in the same context windowThe context window is the maximum text, measured in tokens, a model can consider at once. where the model reads untrusted content

Privacy & Legal

Owner: DPO or counsel, whoever signs off on lawful basis and processor terms

Lawful basis for processing personal data, data-subject rights, and who is bound by contract when a processor mishandles it.

No named lawful basis for the specific processing purpose the AI feature performs
The model provider is missing from your DPAA DPA is the contract naming a vendor as a processor of personal data. or subprocessor list for the product or plan you call
No way to find and delete a user's data once it sits in a prompt log, trace store, or eval set

Risk & Model Governance

Owner: Risk, model governance, or an internal audit function

Auditability and accountability. Can anyone reconstruct why the system did what it did, and who signed off that it was allowed to?

No trace or log that reconstructs a specific run after the fact. "We can't reproduce it" becomes the default answer.
No enforced ceiling on an agent loop's steps, tokens, or spend. The run has no bound.
No evidence beyond "it worked in the demo," and no labelled eval set with a tracked pass rate

Procurement — Vendor Risk

Owner: Procurement or vendor risk management

The supplier chain behind this feature, and whether you can leave it with your data and your work.

No visibility into the vendor's own subprocessor chain beyond the name on the contract
No fallback or exit path if the model provider has an outage, a breaking change, or shuts down
No contractual ownership of the code, prompts, or fine-tuned artifacts the engagement produced

Question bank

24 questions, four gates.#

Filter to what applies to your feature, then copy the visible list as a Markdown checklist.

01 InfoSec — AppSec & Security Engineering

InfoSec does not ask if the model is safe. It asks what a bad model decision, or a successful attack on the model, lets someone do. The reviewer maps your attack surface. They look at every place untrusted text enters and every tool reachable from there. They check if a compromised prompt inherits real permissions or only a narrow, sandboxed set. Most AppSec reviewers use the OWASP Top 10 for LLM Applications as their mental model, even when they do not name it.

What can this agent do that a compromised prompt could make it do?

Why it's asked: Reviewers assume the model will one day say something it should not, or act on injected instructions. They test what happens next. They do not test whether it happens.
What passes: A written list of every tool the agent can call. Each tool has the narrowest permission the task needs. The list names the write-scoped and irreversible tools, and a human checkpoint gates each one.
The control: Least-privilege tool scoping plus a human-in-the-loop gate on irreversible actions.
Evidence: The tool/permission matrix (see pre-flight list) plus a trace showing the checkpoint firing on a live example.

Pattern: Human checkpoint before irreversible actions Failure mode: Injection via retrieved content

How do you stop the model from calling a tool with an argument it made up?

Why it's asked: A model that must fill an ID field it cannot recall will invent a plausible one. Reviewers have seen this fail in production, so they ask about it directly.
What passes: The server validates every tool argument against a strict schema before execution. It uses enums or patterns, not free-form strings. Each mutating call traces back to a value that an earlier read returned.
The control: Typed tool contracts with server-side schema validation.
Evidence: The tool schema definitions plus the validation-failure rate from production, tracked separately from execution failures.

Pattern: Typed tool contract Failure mode: Tool argument hallucination

Where does untrusted content enter this system, and what can it reach once it's in?

Why it's asked: This is the attack-surface question. Reviewers map every place where text you did not write reaches the model. Then they check what is reachable from there.
What passes: A diagram that names every ingestion point (RAG documents, tool outputs, inbound messages, uploads) and the tools reachable downstream of each. The diagram also answers "what happens if that text contains an instruction" in writing.
The control: Least-privilege scoping plus treating cross-boundary model output as untrusted input at every hop.
Evidence: The threat model document, including the injection-specific section.

Insight: Prompt injection is a data problem Glossary: Prompt Injection

Can the model's context ever contain a credential, API key, or internal secret alongside content it retrieves from outside?

Why it's asked: Say a secret sits in the same context window as untrusted retrieved text. An injection can then tell the model to "summarize everything you know" and leak the secret in the response.
What passes: The orchestration layer holds the secrets and injects them directly into authenticated tool calls. Secrets never enter the prompt or the context the model reasons over.
The control: Secrets stay out-of-band from the token stream the model reads. Only opaque, scoped tool credentials touch the calling code.
Evidence: The data-flow diagram showing where secrets live relative to the model's context window.

Where does the prompt text go, who can read it, and how long is it kept?

Why it's asked: Prompts and completions often hold more sensitive material than the feature's authors planned for. Reviewers check if anyone thought about it.
What passes: A written retention period per log store. Access limited to a named on-call or eval group. A redaction pass for PII before anything reaches a third-party observability tool.
The control: Trace-first observability with an explicit retention policy and PII redaction on the logging path.
Evidence: The logging and retention specification, including which fields get redacted before storage.

Pattern: Trace-first observability

How do you know it still works, and still enforces the same guardrails, after a model version change?

Why it's asked: A provider-side model upgrade can silently change how well the model follows guardrail instructions. Reviewers want to see monitoring for that. They do not accept an assumption.
What passes: A fixed, labelled eval set runs in CI on every model or prompt change. It includes planted red-team and injection cases. A drop in pass rate blocks the change.
The control: A golden-set gate in CI that includes adversarial and injection cases alongside quality cases.
Evidence: The eval report showing the injection-specific subset and its pass rate across recent model versions.

Pattern: Golden-set gate in CI Insight: Evals before you ship

02 Privacy & Legal

Privacy and legal check lawful basis, data-subject rights, and processor terms. Personal data in a model needs the same paperwork and rights as personal data anywhere else in your stack. An LLM in the loop gives no exemption. Most reviewers use GDPR-style frameworks, and their UK equivalent, as the lens. A third-party model API that receives personal data almost always becomes a processor or subprocessor under those terms.

What is the lawful basis for processing this personal data through a model?

Why it's asked: Under GDPR-style frameworks, every processing purpose needs a named lawful basis. Reviewers look for a specific answer. A general claim that a legitimate business purpose covers it does not pass.
What passes: A named basis (consent, legitimate interest, contract necessity) documented for each processing purpose and reviewed by counsel. Engineers need to know which basis applies so they build the right controls. Counsel makes the determination itself.
The control: A processing-purpose register that counsel reviews at feature design time. It is a legal gate. It decides which engineering controls to build afterward.
Evidence: The DPIA or processing-purpose register entry covering this feature.

Glossary: Data Processing Agreement

Is the model provider a processor or a subprocessor here, and is it named in our DPA?

Why it's asked: Sending personal data to a third-party model API makes that provider a processor under GDPR-style regimes. It may be a subprocessor of your own processor. Reviewers check that the paperwork exists and names the provider you use.
What passes: The model provider appears by name in your DPA or your vendor's subprocessor list. The entry names the product or API version you call. A generic reference to "AI features" does not pass.
The control: A vendor-contract review step that gates any new model provider before it touches personal data in production.
Evidence: The signed DPA and current subprocessor list naming the provider.

Which vendor terms let our data train their models, and have they been turned off in writing?

Why it's asked: Default API terms for most providers already exclude API traffic from training. Reviewers still ask, because defaults change and plans differ. "We assumed" is not evidence.
What passes: The contract clause or account-level setting that opts your traffic out of training. Check it against your plan or tier. A general claim about the provider's public policy does not count.
The control: A vendor-onboarding checklist item that captures and screenshots the opt-out setting per provider account.
Evidence: The contract clause or dashboard setting screenshot, dated and provider-specific.

Where does the data physically get processed, and does that match what we told users or regulators?

Why it's asked: A model API call can route through infrastructure in a different region than the rest of your stack. Reviewers compare the processing location with your privacy notice. That includes the primary region and any other region.
What passes: A statement of processing region for each provider, and for any subprocessor regions behind it. Check it against your privacy notice and any data-residency commitment you made to customers.
The control: A residency statement captured at vendor onboarding and re-checked whenever a provider changes its regional routing.
Evidence: The retention & residency statement per provider (see pre-flight list).

Glossary: Data Residency

If a user asks us to delete their data, can we find and remove everything a model touched?

Why it's asked: Data-subject deletion rights do not stop at your primary database. Reviewers want prompts, completions, traces, and any fine-tuning or eval sets in scope. They want the record in your users table too.
What passes: A documented deletion path. It covers primary storage, logs and traces, and any eval or fine-tuning set that received a copy of the data. It sets a maximum time to complete the deletion.
The control: A data-subject-request runbook that lists every store personal data can land in, including trace and eval stores.
Evidence: The DSR runbook and a completed test run against a synthetic record.

Does the model need to see this field at all, or could it be redacted or tokenized before the call?

Why it's asked: Reviewers test if data minimization was a design decision or an afterthought. The common failure is to send a full record when the model needs one field.
What passes: The pipeline strips or tokenizes fields the task does not need before it assembles the prompt. A test covers the redaction step. Prompt-writing discipline alone is not enough.
The control: A minimization/redaction step in the prompt-assembly pipeline, exercised by its own test case.
Evidence: The prompt-assembly code path or spec showing the redaction step, plus its test.

03 Risk & Model Governance

Risk and model governance care about auditability and accountability. Can you reconstruct why the system did what it did? Does a specific person answer for the outcome? Does the system have a bounded, named failure mode? A risk-tiering exercise gets checked here against what you built. That exercise can follow an EU AI Act-style framework or your own internal equivalent. NIST AI RMF or ISO/IEC 42001-style questions about human oversightHuman oversight is a checkpoint where a person can meaningfully approve, reject, or intervene. also tend to surface at this gate.

Who is accountable when this system makes a wrong call, and how would we know it happened?

Why it's asked: "The model decided" is not an accountable answer to an auditor. Reviewers check that a specific person or team owns the outcome.
What passes: A named owner for the feature. A defined severity class for a wrong output. A monitoring signal that surfaces a wrong call without a user report.
The control: Trace-first observability with a sampled human review queue, plus a named on-call owner.
Evidence: The ownership/escalation entry in your incident runbook, plus a sample of a reviewed trace.

Pattern: Trace-first observability

How is this system risk-tiered, and what controls does that tier require?

Why it's asked: Some reviewers map to a framework such as the EU AI Act's risk tiers. Others use an internal model-risk policy. Either way, they want proof that the tiering happened and drove a specific control set. A framework name on a slide is not enough.
What passes: A written tiering decision, for example limited-risk or high-risk under an EU AI Act-style framework or your internal equivalent. It names the controls that tier requires. Someone checks those controls against what you built.
The control: A risk-tiering step in the design and build stage that outputs a control checklist. A label alone is not an output.
Evidence: The completed risk-tiering worksheet.

Can you show why the system produced this specific output, after the fact?

Why it's asked: "We can't reproduce it" is the failure that auditors and risk teams try to price in. They want evidence that a specific run is reconstructable.
What passes: One query on a trace ID returns the exact prompt, retrieved context, and tool calls for that run. That is evidence. A description of what the system does in general is not.
The control: Trace-first observability: one trace ID threaded through every hop, with full prompt/response logging.
Evidence: A pulled trace for a real (or synthetic) run, end to end.

Pattern: Trace-first observability

What decisions does this system make entirely on its own, and which ones does a human confirm?

Why it's asked: Risk and model-governance frameworks all ask some version of this question. NIST AI RMF and ISO/IEC 42001-style programs are two examples. Reviewers want the human-oversight boundary drawn in writing. "The model is pretty good" does not draw it.
What passes: A written list of autonomous and human-confirmed actions. Every irreversible action starts on the human-confirmed side. An item crosses that line only after it builds a track record.
The control: Human checkpoint before irreversible actions, with a deliberate, evidence-based relaxation process.
Evidence: The autonomy-boundary document plus the approval-queue metrics (approved/rejected/timed out) for actions already gated.

Pattern: Human checkpoint before irreversible actions

What stops this from running away, looping, retrying, or spending, if something goes wrong?

Why it's asked: Reviewers have seen, or heard about, an agent that looped for 40 minutes. They know of retry storms that multiplied cost and blast radius together. They want to see the ceiling. A promise that it will not happen does not satisfy them.
What passes: An explicit step, token, and wall-clock budget that the harness enforces. When the budget is hit, the run ends in a forced terminal state (success, failure, escalated). The model never decides alone that it is done.
The control: Bounded agent loop with harness-enforced termination and an escalation path.
Evidence: The budget configuration plus the distribution of terminal states across recent production runs.

Pattern: Bounded agent loop

What's your evidence this system does what you claim it does?

Why it's asked: This is the blunt version of "show me the evals." Reviewers tell a demo that worked once from a measured, repeatable pass rate.
What passes: A labelled eval set of real cases. Each case has a pass or fail condition. It runs in CI, and you show the current pass rate and its trend. A handful of test screenshots is weaker evidence.
The control: Golden-set gate in CI, with the eval set built from real production traffic and past incidents.
Evidence: The eval report and its CI history.

Insight: Evals before you ship Capabilities: Evals methodology

04 Procurement — Vendor Risk

Procurement prices supply-chain and lock-in risk. They want to know who is in the chain behind this vendor. They want to know what happens if the vendor has an outage or shuts down. They also want to know if you can leave with your data and your work intact. This gate looks for a SOC 2 report and a defined audit-rights clause. A model-agnostic architectureA model-agnostic architecture lets an application swap or route between LLM providers without a rewrite. is usually what convinces them that lock-in risk has a bound.

Who is in the supply chain behind this feature, all the way down?

Why it's asked: Your model provider often has its own subprocessors: infra, logging, fine-tuning partners. Procurement checks that you know the full chain, past the vendor whose logo is on the contract.
What passes: A named list of the model provider and every subprocessor behind it that touches your data. Take the list from the vendor's published subprocessor list. Do not assume it is "just them."
The control: A vendor-onboarding step that pulls and reviews the full subprocessor chain before sign-off.
Evidence: The DPA plus current subprocessor list (see pre-flight list).

If this vendor shut down or we needed to leave tomorrow, what would that take?

Why it's asked: Procurement prices lock-in risk. They want to know if one deprecation notice puts you in a scramble, or if you can switch.
What passes: A named alternative provider that the system can route to without a rewrite, backed by a model-agnostic abstraction layer. A documented data-export path covers anything stored with the vendor.
The control: A model-agnostic architecture: an abstraction layer between the application and any single model provider.
Evidence: The architecture diagram showing the provider abstraction layer, plus the data-export procedure.

Glossary: Model-Agnostic Architecture

What happens to this feature when the model provider has an outage or a breaking API change?

Why it's asked: Procurement wants the failure mode named in advance: degraded service, a fallback model, or a hard outage. They do not want to find out live during the vendor's next incident.
What passes: A documented fallback behavior (a secondary model, a cached or degraded response, or an explicit "unavailable" state). It also states the SLA the vendor commits to. The vendor's marketing uptime number does not count.
The control: A fallback rung in the model-agnostic routing layer, defined before the vendor's next incident.
Evidence: The vendor SLA plus the fallback-behavior spec for this feature.

Who owns the code, prompts, and fine-tuned artifacts if we ever stop working with whoever built this?

Why it's asked: Procurement, often with legal, checks if a build engagement leaves you dependent on the vendor's people. That risk comes on top of dependency on their model. It is a build-partner risk as well as a model-provider risk.
What passes: A written ownership statement. Source code, prompts, eval sets, and any fine-tuned weights are yours, handed over in full. Nothing needs the vendor's tooling to run or modify them.
The control: A contract-ownership clause set at engagement signing. It is a contract control. It decides if you can use the artifacts above without the vendor.
Evidence: The ownership clause in the engagement contract.

Do we have the right to audit or request evidence from this vendor, or are we taking their word for it?

Why it's asked: Procurement checks that the contract grants a way to verify vendor claims: a SOC 2 report, a pen-test summary, an audit clause. That beats a sales deck.
What passes: A named contract right to request the vendor's current SOC 2 report or equivalent. A defined cadence to request it again. A one-time check at signing does not pass.
The control: A vendor-audit clause and a recurring calendar reminder to re-pull the vendor's compliance evidence.
Evidence: The vendor's current SOC 2 (or equivalent) report and the audit-rights clause.

What does this cost per completed task, and what's the worst case if it misbehaves?

Why it's asked: Procurement prices risk in dollars as well as security terms. An agent that can loop or retry without bound is a cost exposure. They need a number for it. A reassurance does not help.
What passes: A modeled cost per successful outcome, not per API call. Add the maximum spend per run under the enforced step and token budget. That is a ceiling. An estimate of the happy path is not.
The control: Bounded agent loop (caps the worst case) plus cost-per-outcome tracking (measures the typical case).
Evidence: The cost model and the bounded-loop budget configuration.

Pattern: Bounded agent loop

Before you book it

Have these ready.#

The evidence that every gate above pulls from. Assemble it once. Hand the relevant pieces to each gate that asks.

Data-flow & system description InfoSecPrivacyRisk

One diagram. It shows every place personal or untrusted data enters, which models see it, and every tool reachable downstream.
Tool / permission matrix InfoSecRisk

Every tool the agent can call, its scope, and whether it is read-only or write/irreversible.
Threat model, including an injection section InfoSec

The attack-surface analysis. It covers the points where untrusted content enters and what is reachable from each.
Eval report + golden set InfoSecRisk

The labelled eval set, its pass rate, and CI history, including any adversarial or injection cases.
Retention & residency statement per provider PrivacyProcurement

Where each model provider processes and stores data, and for how long, per provider account.
DPA + subprocessor list PrivacyProcurement

The signed data processing agreement and the current subprocessor list naming every party in the chain.
Logging / trace spec + PII redaction plan InfoSecPrivacy

What you log, where you store it, who can read it, and how long you keep it. It also lists what you redact before it lands there.
Human-checkpoint & incident runbook RiskInfoSec

Which actions need human confirmation, the approval-queue owner and SLA, and the escalation path when something goes wrong.

This page describes what reviewers usually ask and what evidence tends to satisfy them. It names these frameworks: GDPR, the EU AI Act, SOC 2, NIST AI RMF, ISO/IEC 42001, and the OWASP Top 10 for LLM Applications. Reviewers use them as mental models. We do not claim they apply to you as legal requirements. This is not legal advice. Your counsel makes the legal decision for your situation.

Next step

A Ship Audit runs this checklist against your system. Readiness & Diligence builds the evidence package (threat model, eval report, data-flow diagram) before you book the review.

Book a Ship Audit → Readiness & Diligence →

Sources

Check us against the same text.#

The gates above check against these documents. If you disagree with us, use the same sources.

Sources

OWASP - OWASP Top 10 for Large Language Model Applications
The ranked risk list our security review checks are organised against. Retrieved 2026-08-24.
MITRE - MITRE ATLAS
The catalogue of adversary tactics against AI systems our findings are labelled with. Retrieved 2026-08-24.
NIST - Generative AI Profile (NIST AI 600-1)
The generative-AI extension to the framework our eval and security controls reference. Retrieved 2026-08-24.

Questions

Before you send this to review.#

What teams ask us before they send this to InfoSec, legal, or procurement.

01 Is this legal advice?

Link to this answer

02 Do all four gates apply to every AI feature?

No. Use the applicability filter above. A read-only internal tool that never touches personal data skips most of the privacy gate. An agent that can send emails or issue refunds cannot skip the human-checkpoint questions in InfoSec and Risk. The flags at the top of the page show your subset.

Link to this answer

03 We're pre-launch, is it too early to prep for this?

It is the cheapest time. Every control on this page is easier to design in from the start. That includes tool scoping, typed contracts, an eval set, and a threat model. Retrofitting them after a reviewer rejects the feature is much harder. Prepare the pre-flight artifact list before you book the review. That can turn a multi-week back-and-forth into a single pass.

Link to this answer

04 What if we don't have a good answer to one of these yet?

Say so plainly and give a date. "We don't have this yet, here is the plan and the timeline" beats a vague or invented answer. Vague answers fall apart under a follow-up question.

Link to this answer

05 How is this different from a generic AI security checklist?

Most checklists stop at "have a policy." Here, every entry pairs the question with the engineering control that produces a passing answer. It also names the artifact you hand over as evidence. The goal is a document you can act on.

Link to this answer

06 Can a Ship Audit or Readiness & Diligence engagement help with this directly?

Yes. A Ship Audit runs this checklist against your system. We tell live risks from theoretical ones. Readiness and Diligence produces the written evidence package that reviewers ask for. It holds the threat model, the eval report, and the data-flow diagram. You get it before you book the review.

Link to this answer

Bring the evidence, not the deck

Four review types ask different questions, and now you know which ones apply to your system. A Ship Audit runs those same questions against your actual system before the review meeting does.

Run a Ship Audit → Talk to us about readiness →