Data Poisoning
Deliberately corrupting a model’s training data so it learns a hidden backdoor or a biased, attacker-chosen behavior.
Data poisoning attacks the supply chain of the model itself rather than its runtime behavior: an attacker who can influence training data (scraped web text, a fine-tuning dataset, human feedback labels) inserts examples designed to teach the model a specific hidden trigger-response pattern, or to subtly bias it. It’s harder to detect than a runtime jailbreak because the compromised behavior looks like ordinary, if slightly unusual, model output until the trigger appears.
Worked example: an attacker injects malicious examples into training/fine-tuning data (or a scraped corpus / RAG source) so the model learns a backdoor or bias — e.g. behaving normally until a trigger phrase appears. Gotcha: it is hard to detect because the model performs well on normal inputs; defenses are provenance/curation of training data, anomaly detection, and for RAG treating retrieved documents as untrusted input (never let a fetched page issue instructions).