Red-Teaming
Deliberately attacking your own model or system to find its failure modes before a real adversary does.
Red-teaming means adopting an attacker’s mindset against your own AI system — trying jailbreaks, prompt injection, edge-case inputs, and adversarial framing — specifically to surface unsafe or embarrassing outputs before shipping. For LLMs it’s become a standard pre-release step precisely because the failure surface (arbitrary natural-language input) is too large to fully enumerate any other way.
Worked example: structured adversarial testing where a team deliberately tries to make a model misbehave — produce disallowed content, leak data, follow injected instructions — before attackers do, to find failure modes happy-path evals miss. Gotcha: it surfaces vulnerabilities but never proves their absence; a model that passes a round is only as safe as the attacks tried, so it is a continuous process (new jailbreaks appear constantly), not a one-time gate, and findings must feed back into training/guardrails to matter.