Jailbreak
A prompt crafted to make a model ignore its safety rules — and the practice of hunting for them.
A jailbreak manipulates a model into violating its own policies — role-play framings, encoding tricks, many-shot patterns, or slow multi-turn escalation. Red-teaming is the defensive discipline: systematically attacking your own system to find these holes before users do. Distinct from prompt injection, which hijacks an application through untrusted content; jailbreaks target the model’s rules directly.
Worked example: a jailbreak is a prompt that coaxes a model past its own safety rules — role-play framings, ‘ignore previous instructions’, or encoding the request — to get output it would normally refuse. Gotcha: jailbreaks (user tricking the model) differ from prompt injection (a third party hiding instructions in data the model reads); defenses need both an instruction hierarchy and output filtering, since neither alone suffices.