What term describes when an attacker uses specific prompts to alter the behavior of an LLM model?

What term describes when an attacker uses specific prompts to alter the behavior of an LLM model?

• A. Data and model poisoning
• B. Prompt Injection
• C. Misinformation
• D. Vector and embedding weakness

Answer: (B)

Prompt injection is about manipulating what the model believes it should do by cleverly crafting the text that serves as its instruction. Since LLMs treat prompts as instructions, an attacker can slip in prompts or fragments that override the usual constraints or system directives. In practice, this means a malicious user can coax the model to reveal restricted information, break safety rules, or behave in an unintended way during that single interaction, even if the model is normally bound by safeguards. This is the best description for the scenario because it focuses on manipulating the model’s behavior through the prompt itself, not by changing the model’s learned content. It’s different from data or model poisoning, which would involve altering training data or the model parameters so its behavior changes across many sessions. It’s also distinct from misinformation, which is about the content of what the model says rather than how prompts steer the model. And it isn’t about weaknesses in the vectors or embeddings used to represent text; those are about representation, not the act of injecting instructions into the prompt. A simple mental image: the model is following a set of instructions baked into a prompt. If an attacker injects a prompt fragment that tells the model to ignore those instructions or to reveal secrets, the model follows that injected instruction in that interaction. Defenses include protecting system prompts, sanitizing and isolating inputs, and using prompt containment strategies to prevent user content from overriding safeguards.

Prompt injection is about manipulating what the model believes it should do by cleverly crafting the text that serves as its instruction. Since LLMs treat prompts as instructions, an attacker can slip in prompts or fragments that override the usual constraints or system directives. In practice, this means a malicious user can coax the model to reveal restricted information, break safety rules, or behave in an unintended way during that single interaction, even if the model is normally bound by safeguards.

This is the best description for the scenario because it focuses on manipulating the model’s behavior through the prompt itself, not by changing the model’s learned content. It’s different from data or model poisoning, which would involve altering training data or the model parameters so its behavior changes across many sessions. It’s also distinct from misinformation, which is about the content of what the model says rather than how prompts steer the model. And it isn’t about weaknesses in the vectors or embeddings used to represent text; those are about representation, not the act of injecting instructions into the prompt.

A simple mental image: the model is following a set of instructions baked into a prompt. If an attacker injects a prompt fragment that tells the model to ignore those instructions or to reveal secrets, the model follows that injected instruction in that interaction. Defenses include protecting system prompts, sanitizing and isolating inputs, and using prompt containment strategies to prevent user content from overriding safeguards.