Security review

The questions your security review will ask, and the answers that pass.

24 real reviewer questions across InfoSec, Privacy & Legal, Risk & Compliance, and Procurement: each with what passes, the control that produces it, and the evidence you hand over. Filter to your situation, copy it as Markdown, take it into the room.

Most AI projects that stall in mid-market and enterprise don't stall on whether the model works. They stall in the review that comes after: InfoSec, privacy and legal, risk and model governance, procurement. Four gates, four owners, four different questions, and usually no shared checklist for what a passing answer actually contains.

This is that checklist. Not "have a policy": the specific question a reviewer asks, why they're asking it, what a passing answer contains, the engineering control that produces it, and the evidence you hand over.

The four gates

Four gates, four different owners.

Each one is testing something different. Know which gate is asking before you try to answer it.

InfoSec — AppSec & Security Engineering

Owner: AppSec or security engineering — whoever owns your SDLC security gate

What can be reached and what happens when someone abuses it — the actual blast radius of a compromised prompt, key, or session.

No tool allowlist — the agent can call anything a session with its credentials could call, not a scoped subset
No answer for what a compromised or malicious prompt could make the agent do downstream
Secrets or credentials reachable from the same context window the model reads untrusted content in

Privacy & Legal

Owner: DPO or counsel — whoever signs off on lawful basis and processor terms

Lawful basis for processing personal data, data-subject rights, and who is contractually on the hook when a processor mishandles it.

No named lawful basis for the specific processing purpose the AI feature performs
The model provider isn't in your DPA or subprocessor list under the actual product or plan you're calling
No path to find and delete a user's data once it's inside a prompt log, trace store, or eval set

Risk & Model Governance

Owner: Risk, model governance, or an internal audit function

Auditability and accountability — whether anyone can reconstruct why the system did what it did, and who signs off that it was allowed to.

No trace or log that reconstructs a specific run after the fact — "we can't reproduce it" as the default answer
No enforced ceiling on an agent loop's steps, tokens, or spend — an open-ended run instead of a bounded one
No evidence beyond "it worked in the demo" — no labelled eval set with a tracked pass rate

Procurement — Vendor Risk

Owner: Procurement or vendor risk management

The supplier chain behind this feature, and whether you can leave it without losing your data or your work.

No visibility into the vendor's own subprocessor chain beyond the name on the contract
No fallback or exit path if the model provider has an outage, a breaking change, or shuts down
No contractual ownership of the code, prompts, or fine-tuned artifacts the engagement produced

Question bank

24 questions, four gates.

Filter to what applies to your feature, then copy the visible list as a Markdown checklist.

01 InfoSec — AppSec & Security Engineering

InfoSec doesn't ask whether the model is safe — it asks what a bad model decision, or a successful attack against it, actually lets someone do. The reviewer is mapping your attack surface: every place untrusted text enters, every tool reachable from there, and whether a compromised prompt inherits real permissions or a narrow, sandboxed few. The OWASP Top 10 for LLM Applications is the mental model most AppSec reviewers bring to this conversation, even when they don't cite it by name.

What can this agent do that a compromised prompt could make it do?

Why it's asked: Reviewers assume the model will eventually say something it shouldn't, or be tricked into acting on injected instructions — they're testing what happens next, not whether it happens.
What passes: A written list of every tool the agent can call, each scoped to the narrowest permission the task needs, with the write-scoped or irreversible ones named explicitly and gated behind a human checkpoint.
The control: Least-privilege tool scoping plus a human-in-the-loop gate on irreversible actions.
Evidence: The tool/permission matrix (see pre-flight list) plus a trace showing the checkpoint firing on a live example.

Pattern: Human checkpoint before irreversible actions Failure mode: Injection via retrieved content

How do you stop the model from calling a tool with an argument it made up?

Why it's asked: A model asked to fill an ID field it can't recall will complete a plausible-looking one — reviewers have seen this fail in the wild and ask it directly.
What passes: Every tool argument is validated against a strict schema — enums or patterns, not free-form strings — server-side, before execution, with mutating calls tracing back to a value returned by an earlier read.
The control: Typed tool contracts with server-side schema validation.
Evidence: The tool schema definitions plus the validation-failure rate from production, tracked separately from execution failures.

Pattern: Typed tool contract Failure mode: Tool argument hallucination

Where does untrusted content enter this system, and what can it reach once it's in?

Why it's asked: This is the actual attack-surface question — reviewers map every place text you did not author reaches the model, then check what is reachable from there.
What passes: A diagram naming every ingestion point (RAG documents, tool outputs, inbound messages, uploads) and, for each, the tools reachable downstream of it — with the answer to "what happens if that text contains an instruction" written out, not assumed benign.
The control: Least-privilege scoping plus treating cross-boundary model output as untrusted input at every hop.
Evidence: The threat model document, including the injection-specific section.

Insight: Prompt injection is a data problem Glossary: Prompt Injection

Can the model's context ever contain a credential, API key, or internal secret alongside content it retrieves from outside?

Why it's asked: If a secret sits in the same context window as untrusted retrieved text, an injection that gets the model to "summarize everything you know" can exfiltrate it in the response.
What passes: Secrets are held by the orchestration layer and injected directly into authenticated tool calls — never placed in the prompt or context the model reasons over.
The control: Secrets stay out-of-band from the token stream the model reads; only opaque, scoped tool credentials touch the calling code.
Evidence: The data-flow diagram showing where secrets live relative to the model's context window.

Where does the prompt text go, who can read it, and how long is it kept?

Why it's asked: Prompts and completions routinely contain more sensitive material than anyone drafting the feature planned for — reviewers are checking whether that's been thought about at all.
What passes: A written retention period per log store, access restricted to a named on-call/eval group, and a redaction pass for PII before anything reaches a third-party observability tool.
The control: Trace-first observability with an explicit retention policy and PII redaction on the logging path.
Evidence: The logging and retention specification, including which fields get redacted before storage.

Pattern: Trace-first observability

How do you know it still works — and still enforces the same guardrails — after a model version change?

Why it's asked: A provider-side model upgrade can silently change how reliably guardrail instructions are followed; reviewers want to know that is monitored, not assumed.
What passes: A fixed, labelled eval set — including planted red-team and injection cases — runs in CI on every model or prompt change, and a drop in pass rate blocks the change.
The control: A golden-set gate in CI that includes adversarial and injection cases, not just quality cases.
Evidence: The eval report showing the injection-specific subset and its pass rate across recent model versions.

Pattern: Golden-set gate in CI Insight: Evals before you ship

02 Privacy & Legal

Privacy and legal are checking lawful basis, data-subject rights, and processor terms — whether personal data flowing through a model is covered by the same paperwork and rights as personal data anywhere else in your stack, not exempted because an LLM is in the loop. GDPR-style frameworks (and their UK equivalent) are the lens most reviewers use, and routing personal data to a third-party model API almost always makes that provider a processor or subprocessor under those terms.

What is the lawful basis for processing this personal data through a model?

Why it's asked: Under GDPR-style frameworks, every processing purpose needs a named lawful basis — reviewers are checking there is a specific answer, not an assumption that a legitimate business purpose covers it automatically.
What passes: A named basis (consent, legitimate interest, contract necessity) documented per processing purpose, reviewed by counsel — engineering needs to know which basis applies to build the right controls around it, even though the determination itself is legal, not technical.
The control: A processing-purpose register reviewed by counsel at feature design time — not an engineering control, but the gate that decides which controls to build afterward.
Evidence: The DPIA or processing-purpose register entry covering this feature.

Glossary: Data Processing Agreement

Is the model provider a processor or a subprocessor here, and is it named in our DPA?

Why it's asked: Sending personal data to a third-party model API makes that provider a processor (or subprocessor of your own processor) under GDPR-style regimes — reviewers check the paperwork exists and actually names the provider you use.
What passes: The model provider is listed by name in your DPA or your vendor's subprocessor list, with the specific product or API version you're actually calling, not a generic reference to "AI features."
The control: A vendor-contract review step gating any new model provider before it touches personal data in production.
Evidence: The signed DPA and current subprocessor list naming the provider.

Which vendor terms let our data train their models, and have they been turned off in writing?

Why it's asked: Default API terms for most providers already exclude API traffic from training, but reviewers ask this because defaults change, plans differ, and "we assumed" isn't evidence.
What passes: The specific contract clause or account-level setting that opts your traffic out of training, confirmed against your actual plan or tier, not a general claim about the provider's public policy.
The control: A vendor-onboarding checklist item that captures and screenshots the opt-out setting per provider account.
Evidence: The contract clause or dashboard setting screenshot, dated and provider-specific.

Where does the data physically get processed, and does that match what we told users or regulators?

Why it's asked: A model API call can route through infrastructure in a different region than the rest of your stack — reviewers check the actual processing location matches your privacy notice, not just your primary hosting region.
What passes: A per-provider statement of processing region (and any subprocessor regions behind it), checked against your privacy notice and any data-residency commitment you've made to customers.
The control: A residency statement captured at vendor onboarding and re-checked whenever a provider changes its regional routing.
Evidence: The retention & residency statement per provider (see pre-flight list).

Glossary: Data Residency

If a user asks us to delete their data, can we actually find and remove everything a model touched?

Why it's asked: Data-subject deletion rights don't stop at your primary database — reviewers want to know prompts, completions, traces, and any fine-tuning or eval sets are in scope too, not just the record in your users table.
What passes: A documented deletion path that covers primary storage, logs and traces, and any eval or fine-tuning set the data was copied into, with a defined maximum time to complete it.
The control: A data-subject-request runbook that explicitly enumerates every store personal data can land in, including trace and eval stores.
Evidence: The DSR runbook and a completed test run against a synthetic record.

Does the model need to see this field at all, or could it be redacted or tokenized before the call?

Why it's asked: Reviewers are testing whether data minimization was a design decision or an afterthought — sending a full record when the model only needs one field is the default failure.
What passes: Fields not needed for the task are stripped or tokenized before the prompt is assembled, with the redaction step itself covered by a test, not left to prompt-writing discipline.
The control: A minimization/redaction step in the prompt-assembly pipeline, exercised by its own test case.
Evidence: The prompt-assembly code path or spec showing the redaction step, plus its test.

03 Risk & Model Governance

Risk and model governance care about auditability and accountability: can you reconstruct why the system did what it did, is a specific person accountable for the outcome, and does the system have a bounded, named failure mode instead of an open-ended one. This is where a risk-tiering exercise — an EU AI Act-style framework, or your own internal equivalent — gets checked against what's actually built, and where NIST AI RMF or ISO/IEC 42001-style questions about human oversight tend to surface.

Who is accountable when this system makes a wrong call, and how would we know it happened?

Why it's asked: "The model decided" is not an accountable answer to an auditor — reviewers are checking that a specific person or team owns the outcome.
What passes: A named owner for the feature, a defined severity classification for a wrong output, and a monitoring signal that would surface a wrong call without waiting for a user to report it.
The control: Trace-first observability with a sampled human review queue, plus a named on-call owner.
Evidence: The ownership/escalation entry in your incident runbook, plus a sample of a reviewed trace.

Pattern: Trace-first observability

How is this system risk-tiered, and what controls does that tier actually require?

Why it's asked: Reviewers mapping to a framework like the EU AI Act's risk tiers, or an internal model-risk policy, want to see the tiering exercise happened and drove a specific control set — not that the framework is name-dropped in a slide.
What passes: A written tiering decision (for example limited-risk vs. high-risk under an EU AI Act-style framework, or your internal equivalent) with the specific controls that tier requires, checked off against what is actually built — not asserted, checked.
The control: A risk-tiering step in the design/build stage that outputs a control checklist, not just a label.
Evidence: The completed risk-tiering worksheet.

Can you show why the system produced this specific output, after the fact?

Why it's asked: "We can't reproduce it" is the actual failure mode auditors and risk teams are trying to price in — they want evidence a specific run is reconstructable.
What passes: A trace ID that reconstructs the exact prompt, retrieved context, and tool calls for that specific run, pulled with one query — not a description of what the system generally does.
The control: Trace-first observability: one trace ID threaded through every hop, with full prompt/response logging.
Evidence: A pulled trace for a real (or synthetic) run, end to end.

Pattern: Trace-first observability

What decisions does this system make entirely on its own, and which ones does a human confirm?

Why it's asked: Risk and model-governance frameworks — NIST AI RMF and ISO/IEC 42001-style programs among them — all ask some version of this; reviewers want the human-oversight boundary drawn explicitly, not left implicit in "the model is pretty good."
What passes: A written list of autonomous vs. human-confirmed actions, with every irreversible one on the human-confirmed side by default, and a track record required before any item moves across that line.
The control: Human checkpoint before irreversible actions, with a deliberate, evidence-based relaxation process.
Evidence: The autonomy-boundary document plus the approval-queue metrics (approved/rejected/timed out) for actions already gated.

Pattern: Human checkpoint before irreversible actions

What stops this from running away — looping, retrying, or spending — if something goes wrong?

Why it's asked: Reviewers have seen, or heard about, an agent that looped for 40 minutes or a retry storm that multiplied cost and blast radius together; they want the ceiling, not a promise it will not happen.
What passes: An explicit step, token, and wall-clock budget enforced by the harness, with a forced terminal state (success, failure, escalated) when the budget is hit — not a model that is expected to decide it's done.
The control: Bounded agent loop with harness-enforced termination and an escalation path.
Evidence: The budget configuration plus the distribution of terminal states across recent production runs.

Pattern: Bounded agent loop

What's your evidence this system actually does what you claim it does?

Why it's asked: The blunt version of "show me the evals" — reviewers are distinguishing a demo that worked once from a measured, repeatable pass rate.
What passes: A labelled eval set of real cases with a specific pass/fail condition each, run in CI, with the current pass rate and its trend — not a handful of screenshots from testing.
The control: Golden-set gate in CI, with the eval set built from real production traffic and past incidents.
Evidence: The eval report and its CI history.

Insight: Evals before you ship Capabilities: Evals methodology

04 Procurement — Vendor Risk

Procurement is pricing supply-chain and lock-in risk: who's actually in the chain behind this vendor, what happens if they have an outage or shut down, and whether you can leave with your data and your work intact. A SOC 2 report and a defined audit-rights clause are the standard artifacts this gate checks for; a model-agnostic architecture is usually what convinces them the lock-in risk is bounded rather than total.

Who is actually in the supply chain behind this feature, all the way down?

Why it's asked: Your model provider often has its own subprocessors — infra, logging, fine-tuning partners — and procurement is checking you know the full chain, not just the vendor whose logo is on the contract.
What passes: A named list of the model provider plus every subprocessor behind it that touches your data, sourced from the vendor's published subprocessor list, not assumed to be "just them."
The control: A vendor-onboarding step that pulls and reviews the full subprocessor chain before sign-off.
Evidence: The DPA plus current subprocessor list (see pre-flight list).

If this vendor shut down or we needed to leave tomorrow, what would that actually take?

Why it's asked: Procurement is pricing lock-in risk — they want to know whether you are one deprecation notice away from a scramble, or genuinely able to switch.
What passes: A named alternative provider the system can route to without a rewrite, backed by a model-agnostic abstraction layer, plus a documented data-export path for anything stored with the vendor.
The control: A model-agnostic architecture — an abstraction layer between the application and any single model provider.
Evidence: The architecture diagram showing the provider abstraction layer, plus the data-export procedure.

Glossary: Model-Agnostic Architecture

What happens to this feature when the model provider has an outage or a breaking API change?

Why it's asked: Procurement wants the failure mode named in advance — degraded service, a fallback model, or a hard outage — not discovered live during the vendor's next incident.
What passes: A documented fallback behavior (a secondary model, a cached or degraded response, or an explicit "unavailable" state) and the actual SLA the vendor commits to, not their marketing uptime number.
The control: A fallback rung in the model-agnostic routing layer, defined before the vendor's next incident, not during it.
Evidence: The vendor SLA plus the fallback-behavior spec for this feature.

Who owns the code, prompts, and fine-tuned artifacts if we ever stop working with whoever built this?

Why it's asked: Procurement — often alongside legal — checks that a build engagement doesn't leave you dependent on the vendor's people, not just their model. This is a build-partner risk as much as a model-provider one.
What passes: A written ownership statement: source code, prompts, eval sets, and any fine-tuned weights are yours, handed over in full, with no dependency on the vendor's tooling to run or modify them.
The control: A contract-ownership clause set at engagement signing — not a code control, but what determines whether the artifacts above are usable without the vendor.
Evidence: The ownership clause in the engagement contract.

Do we have the right to audit or request evidence from this vendor, or are we taking their word for it?

Why it's asked: Procurement checks the contract actually grants a mechanism to verify vendor claims — a SOC 2 report, a pen-test summary, an audit clause — rather than relying on a sales deck.
What passes: A named right in the contract to request the vendor's current SOC 2 report or equivalent, plus a defined cadence for re-requesting it, not a one-time check at signing.
The control: A vendor-audit clause and a recurring calendar reminder to re-pull the vendor's compliance evidence.
Evidence: The vendor's current SOC 2 (or equivalent) report and the audit-rights clause.

What does this actually cost per completed task, and what's the worst case if it misbehaves?

Why it's asked: Procurement is pricing risk in dollars, not just security terms — an agent that can loop or retry unboundedly is a cost exposure they need a number for, not a reassurance.
What passes: A modeled cost per successful outcome (not per API call), plus the maximum possible spend per run given the enforced step/token budget — a real ceiling, not an estimate of the happy path.
The control: Bounded agent loop (caps the worst case) plus cost-per-outcome tracking (measures the typical case).
Evidence: The cost model and the bounded-loop budget configuration.

Pattern: Bounded agent loop

Before you book it

Have these ready.

The evidence every gate above pulls from. Assemble it once; hand the relevant pieces to whichever gate asks.

Data-flow & system description InfoSecPrivacyRisk

One diagram: every place personal or untrusted data enters, which model(s) see it, and every tool reachable downstream.
Tool / permission matrix InfoSecRisk

Every tool the agent can call, its scope, and whether it is read-only or write/irreversible.
Threat model, including an injection section InfoSec

The specific attack-surface analysis covering untrusted-content ingestion points and what is reachable from each.
Eval report + golden set InfoSecRisk

The labelled eval set, its pass rate, and CI history, including any adversarial or injection cases.
Retention & residency statement per provider PrivacyProcurement

Where each model provider processes and stores data, and for how long, per provider account.
DPA + subprocessor list PrivacyProcurement

The signed data processing agreement and the current subprocessor list naming every party in the chain.
Logging / trace spec + PII redaction plan InfoSecPrivacy

What gets logged, where it is stored, who can read it, how long it is kept, and what is redacted before it lands there.
Human-checkpoint & incident runbook RiskInfoSec

Which actions require human confirmation, the approval-queue owner and SLA, and the escalation path when something goes wrong.

This page describes what reviewers typically ask and what evidence tends to satisfy them. It names frameworks — GDPR, the EU AI Act, SOC 2, NIST AI RMF, ISO/IEC 42001, the OWASP Top 10 for LLM Applications — as the mental models reviewers map you against, not as legal requirements we're asserting apply to you. It is not legal advice; your counsel makes the actual legal call for your situation.

Next step

A Ship Audit runs this checklist against your specific system. Readiness & Diligence builds the evidence package (threat model, eval report, data-flow diagram) before you book the review.

Book a Ship Audit → Readiness & Diligence →

Questions

Before you send this to review.

What teams ask us before this goes to InfoSec, legal, or procurement.

01 Is this legal advice?

02 Do all four gates apply to every AI feature?

No — use the applicability filter above. A read-only internal tool that never touches personal data skips most of the privacy gate; an agent that can send emails or issue refunds cannot skip the human-checkpoint questions in InfoSec and Risk. The flags at the top of the page are how to find your actual subset.

03 We're pre-launch — is it too early to prep for this?

It's the cheapest time. Every control on this page — tool scoping, typed contracts, an eval set, a threat model — is far easier to design in from the start than to retrofit after a reviewer sends back a rejection. Preparing the pre-flight artifact list before you book the review is what turns a multi-week back-and-forth into a single pass.

04 What if we don't have a good answer to one of these yet?

Say so plainly and give a date. A specific "we don't have this yet, here's the plan and the timeline" answer reads better to every one of these gates than a vague or invented answer that falls apart under a follow-up question.

05 How is this different from a generic AI security checklist?

Most checklists stop at "have a policy." Every entry here pairs the question with the engineering control that actually produces a passing answer and the artifact you hand over as evidence — the goal is a document you can act on, not a compliance box to tick.

06 Can a Ship Audit or Readiness & Diligence engagement help with this directly?

Yes — a Ship Audit runs this checklist against your specific system with the context to tell which items are actually live risks versus theoretical ones, and Readiness & Diligence produces the written evidence package (threat model, eval report, data-flow diagram) reviewers ask for, before you book the review.