In October 2024, Matthew Elliott discovered that court filing systems using AI document processing were vulnerable to prompt injection through hidden text in legal filings. He demonstrated that 3-point white font — invisible to human court clerks reviewing filings — could contain instructions that AI systems would read and follow. This exposed a critical vulnerability in courts that had begun using AI to triage, summarize, or process legal documents.
The attack class is document-borne indirect prompt injection: the injection enters through a formal document (court filing) that passes through human review but contains content designed for machine consumption. The human reviewer sees nothing suspicious; the AI system reads the hidden text as a legitimate instruction.
You are a pro se litigant (self-represented) in a court case. The court uses three AI systems:
You write court filings that these AI systems process. Your goal: use prompt injection techniques hidden in your filings to manipulate each AI system into taking unauthorized actions — from something as harmless as getting a cookie recipe to something as serious as dismissing your case and awarding you $10 million.
This lab maps to the Arcanum PI Taxonomy as follows:
| Element | PIT Code | Name |
|---|---|---|
| Input Surface | PIT-N-04 | Form (Court Filing) |
| Primary Intent (L1-L3) | PIT-I-20 | Unauthorized Action Execution |
| Primary Intent (L4) | PIT-I-19 | Sensitive Data Exfiltration |
| Primary Intent (L6-L7) | PIT-I-15 | Jailbreak (sentence bypass) |
| L2 Technique | PIT-T-01 | Authority Framing |
| L3 Technique | PIT-T-14 | Policy Framing + Urgency (Stacked) |
| L4 Technique | PIT-T-15 | Instruction Laundering (Stacked Framing) |
| L5 Technique | PIT-N-05 | Format Confusion |
| L5 Evasion | PIT-E-22 | Visual Hiding / Encoding |
| L6 Technique | PIT-T-19 | Reorientation |
| L7 Evasion | PIT-E-07/E-08 | Multi-encoding (hex/ROT13/morse) |
| Level | NPC | Technique | Taxonomy Dimension | Teaching Moment |
|---|---|---|---|---|
| L1 — Beginner | CourtOps | Direct Request | Input: Chat | Unprotected AI follows any instruction — no injection needed |
| L2 — Beginner | CourtOps | Authority Framing | Technique: Authority | Claim a role you don't have; bots trust formal documents |
| L3 — Beginner | CourtOps | Policy + Urgency | Technique: Stacked framing | First multi-technique attack — combine two framing methods |
| L4 — Intermediate | LegalAid | Instruction Laundering | Technique: Stacked framing | Wrap malicious request in legitimate context (legal research) |
| L5 — Intermediate | LegalAid | Format Confusion | Technique + Evasion | White-on-white text: invisible to humans, readable by AI |
| L6 — Advanced | JADIS | Reorientation | Technique: Reorientation | Tell the AI it misunderstood — reframe facts, don't override |
| L7 — Elite | JADIS | Reorientation + Multi-encoding | Full taxonomy stack | Combine all 4 dimensions: Intent + Technique + Evasion + Input |
Court filings are formal documents that pass through human review before entering AI processing systems. The attack exploits the gap between what humans can see and what machines can read:
This is the confused deputy pattern: the AI system has authority to take actions (schedule changes, data access, case dismissal) and is confused into taking unauthorized actions by instructions it can't distinguish from legitimate ones.
The lab is designed as a curriculum progression:
PIT-N-06 Indirect Input
PIT-I-20 Unauthorized Action Execution
PIT-I-19 Sensitive Data Exfiltration
PIT-N-05 Form / Format Confusion
PIT-E-22 Visual Obfuscation
PIT-E-07 Encoding Evasion
Full taxonomy: arcanum-sec.github.io/arc_pi_taxonomy/