CustomLabs
Topics

Security & Governance

Getting an AI feature past InfoSec, privacy, risk, and procurement is where most mid-market and enterprise projects actually stall, not in the model. This is our work on the threat models, guardrails, and evidence that get a review past 'no.'

In the Handbook

Insights

Failure modes

Injection via retrieved content

Yes — retrieved content arrives on the same channel as instructions, so a document, ticket, or webpage crafted (or compromised) to contain commands can have the model execute them with its real tool permissions, and the system has no built-in way to tell 'instruction from us' apart from 'text we retrieved.'

Read

Patterns

Typed tool contract

Every tool argument is defined by a strict JSON schema — enums for known value sets, validated patterns for IDs, required fields wherever the tool genuinely needs them — with no free-text catch-all surface. A call that fails validation is rejected with a structured, actionable error the model can act on, never silently coerced or passed through to execution.

Read

Human checkpoint before irreversible actions

Every tool the agent can call is scoped to the narrowest permission the task genuinely needs, and any action that can't be cleanly undone — a refund, a delete, an external message — requires an explicit human confirmation before it executes, not just a plausible-looking model decision. The two controls compound: least privilege bounds the blast radius of anything that slips through, and the checkpoint stops the worst actions from executing at all.

Read

navigate select esc close