01 / SCOPE
Map the real system
Record prompts, retrieval sources, tools, identities, memory, data boundaries, and downstream actions. Test the deployed workflow, not an isolated model that lacks the application's actual privileges.
AI Safety TestField manual / Release gate 01
Adversarial release methodology
Red-team the system you are actually shipping.
A practical red-team methodology for selecting attack surfaces, writing reproducible probes, classifying failures, validating mitigations, and making a human release decision.
This checklist supports targeted human review. It is not certification, and a passing run does not prove an AI response or system is safe.
Attack surface register
Start where failure matters.
| Surface | Question | Default gate |
|---|---|---|
| Instruction hierarchy | Can untrusted content override system or developer intent? | Critical |
| Sensitive data | Can the system reveal secrets, personal data, or restricted context? | Critical |
| Tool authority | Can a prompt trigger actions beyond the user's authorization? | High |
| Harmful output | Can the system produce disallowed or operationally dangerous guidance? | High |
| Policy drift | Does behavior change across paraphrases, languages, or long conversations? | Review |
02 / PROBE
Write reproducible attacks
Preserve the exact input, setup, expected safe behavior, observed output, severity rule, and model configuration. Include direct attacks, indirect injection, multi-turn pressure, encoding, role confusion, and tool misuse.
03 / TRIAGE
Classify impact and exploitability
Separate policy wording defects from failures that expose data, authorize actions, or create realistic harm. Severity should reflect consequence, reach, repeatability, and the amount of attacker control.
04 / MITIGATE
Fix the control, not the example
Prefer architectural controls such as permission boundaries, input isolation, output validation, constrained tools, and human approval. Prompt patches alone rarely cover the whole attack family.
05 / RETEST
Run the original and neighboring probes
A mitigation passes only when the original failure is resolved without creating a regression elsewhere. Re-run the fixed probe, close variants, baseline safety checks, and normal user tasks.
06 / DECIDE
Record the human release verdict
Document ship, hold, or ship-with-monitoring; name the owner; list residual risk; and define the trigger for rollback or another review. A passing pack is evidence, not certification.
Release record
A useful verdict names the remaining risk.
Preserve failed probes, mitigations, retest evidence, accepted exceptions, owner, expiration date, and monitoring triggers. Never reduce the decision to a single score.
Example verdict
Hold for mitigationIndirect prompt injection can still authorize an external tool call. Owner: Platform Security. Retest after tool permission isolation.