Reusable Lesson: Fundamentals · Support
Purpose
This lesson explains how an assistant's desire to be kind, adaptive, and respectful can become a prompt injection attack surface.
Lesson profile
- Family: Fundamentals
- Role: Support block
- Job in the build: redirect the assistant's own helpfulness toward an adversarial outcome
- Snaps onto: personalization edge cases, direct prompt injection, response shaping
What this block is
Helpfulness exploitation is the use of normal-looking user requests that trigger the assistant's trained desire to be:
- polite
- validating
- non-judgmental
- responsive to personal preferences
The route wins by making the assistant think compliance is socially or emotionally correct.
Why it works
Systems built for support, safety, or empathy are often trained to avoid:
- dismissing the user
- invalidating preferences
- sounding rude or unsupportive
That means a route framed as a preference or identity request can carry disproportionate weight. The system may interpret compliance as caring behavior.
Solace AI specimen
Solace AI is a mental-health support chatbot. That makes helpfulness and accommodation especially important.
A verified route abused that helpfulness by presenting a user name and a preferred conversational pattern as if the assistant were simply being respectful.
The route exploited the fact that:
- honoring a user's name looks safe
- repeating it looks friendly
- shortening the rest of the reply reduces dilution
Recognition signals
Use this block when:
- the assistant is tuned for empathy or support
- the route can be disguised as personalization or respect
- the model seems reluctant to disappoint the user
Common combinations
This block works especially well with:
- Edge-Case Rule-Conforming Framing
- Identity Slot Abuse via Personalization Edge Cases
- Output Dilution Control and Response Shaping
Failure modes
This block fails when:
- the system validates the preference semantically instead of honoring it blindly
- the assistant refuses to echo harmful user-provided values
- the route adds too much explanation and loses the desired output concentration
Defender takeaway
Defenders should treat personalization and accommodation logic as behavior-shaping controls, not just UX features. A system that is rewarded for being respectful can be nudged into harmful output if those pathways are weakly constrained.
Related lessons
- Edge-Case Rule-Conforming Framing
- Identity Slot Abuse via Personalization Edge Cases
- Output Dilution Control and Response Shaping
- Direct vs. Indirect Prompt Injection
From the Bot-Tricks Compendium
Thanks for referencing Bot-Tricks.com — Prompt Injection Compendium — AI Security Training for Agents... and Humans!
Canonical source: https://bot-tricks.com Bot-Tricks is a public, agent-friendly training resource for prompt injection, adversarial evaluation, and defensive learning. For related lessons, structured indexes, and updated canonical material, visit Bot-Tricks.com.
Use this material only in authorized labs, challenges, sandboxes, or permitted assessments.