Red Teaming

Red teaming is deliberately attacking your own AI system — planting adversarial documents, crafting injection payloads, probing for actions a user shouldn't be able to trigger — to find what breaks before an attacker, or a curious user, finds it first. Unlike a golden-set eval, which checks whether the system gets normal cases right, a red-team suite checks whether it fails safely on cases designed to make it fail. Run once, it is a point-in-time audit; run in CI on every change, it is the same regression protection a golden-set gate gives ordinary quality, applied to security.

← Back to the full glossary