Which term describes risk where data used to train or validate the model can be manipulated to embed backdoors or biases?

Which term describes risk where data used to train or validate the model can be manipulated to embed backdoors or biases?

• A. Improper output handling
• B. Data and model poisoning
• C. Unbounded consumption
• D. System prompt leakage

Answer: (B)

Data poisoning is the risk where the data used to train or validate a model is tampered with to embed backdoors or biases. By injecting carefully crafted examples into the training set, an attacker can make the model learn hidden behaviors that only activate under specific triggers, effectively creating a backdoor. Poisoned validation data can also skew evaluation results and encourage biased or unsafe model behavior once deployed, because the model has learned from misleading signals. Other terms describe different issues: improper output handling concerns how outputs are managed or sanitized, unbounded consumption relates to resource or data ingestion limits, and system prompt leakage involves hidden instructions or prompts leaking into the model’s behavior.

Data poisoning is the risk where the data used to train or validate a model is tampered with to embed backdoors or biases. By injecting carefully crafted examples into the training set, an attacker can make the model learn hidden behaviors that only activate under specific triggers, effectively creating a backdoor. Poisoned validation data can also skew evaluation results and encourage biased or unsafe model behavior once deployed, because the model has learned from misleading signals. Other terms describe different issues: improper output handling concerns how outputs are managed or sanitized, unbounded consumption relates to resource or data ingestion limits, and system prompt leakage involves hidden instructions or prompts leaking into the model’s behavior.