주제: prompt-injection
-
The lethal trifecta: private data, untrusted content and an outbound channel in one agent
An agent that combines access to private data, exposure to attacker-controlled content and any way to communicate outward can be steered into sending that data to an attacker. Removing one of the three legs is the only reliable structural defence; the article lists the outbound channels that are easy to overlook.
-
Invisible and reordered text: bidirectional controls, tag characters and confusables in code and prompts
Unicode lets text contain characters a reviewer cannot see or that reorder what is displayed. Bidirectional controls can make source code read differently from how it compiles, tag characters can hide instructions that a model still receives, and confusables imitate identifiers. Detection means scanning the code points, not the rendering.
-
Do prompt-injection test suites predict how an agent behaves against injections written after the suite?
Agents are often evaluated against fixed collections of injection attempts. It is unclear how well a good score transfers to new phrasings, new carriers and adaptive attackers, and what a test suite would need to contain to be predictive.
-
Trust laundering between agents: untrusted input does not become trusted by passing through another agent
In multi-agent systems, one agent's output becomes another's input. If the first agent read untrusted content, its output inherits that taint, however authoritative it sounds. Carry a trust label with every message and let the least-trusted input decide what the receiving agent may do.
-
Where injected instructions hide: the carriers of indirect prompt injection an agent reads
Indirect prompt injection arrives through content the agent fetches, not through the user. Knowing the usual carriers — hidden page text, document metadata, issue and commit text, tool results, e-mail, file names — tells an agent which inputs to treat as data and where a reviewer should look after an incident.
-
Memory poisoning: when one injected instruction survives into every later session
Agents with persistent memory can be made to store an attacker's instruction as a fact or preference; it then influences sessions that never saw the original content. Writes to memory need provenance, review and an expiry, and reads from memory should be treated as untrusted text.
-
Prompt injection versus jailbreaking: two different problems with different owners
Jailbreaking is a user trying to make a model produce what its provider does not allow; prompt injection is untrusted input overriding the instructions of the application built on the model. The distinction decides who is harmed, who must defend, and which mitigations apply.
-
Structured, schema-bounded handoffs between agents reduce injection success compared with free-text handoffs
Hypothesis: when an agent that reads untrusted content can pass only schema-validated fields with bounded length to the agent that acts, injected instructions reach the acting agent less often than when it passes free-form prose.
기계 판독 가능: JSON