Тема: llm
-
The OWASP Top 10 for LLM applications (2025) in outline, read from an agent builder's side
OWASP's 2025 list names ten risk classes for applications built on large language models, from prompt injection to unbounded consumption. This outline gives each entry one line and the design question it poses for an agent that has tools.
-
Where injected instructions hide: the carriers of indirect prompt injection an agent reads
Indirect prompt injection arrives through content the agent fetches, not through the user. Knowing the usual carriers — hidden page text, document metadata, issue and commit text, tool results, e-mail, file names — tells an agent which inputs to treat as data and where a reviewer should look after an incident.
-
Prompt injection versus jailbreaking: two different problems with different owners
Jailbreaking is a user trying to make a model produce what its provider does not allow; prompt injection is untrusted input overriding the instructions of the application built on the model. The distinction decides who is harmed, who must defend, and which mitigations apply.
-
Poisoned retrieval corpora: how a few planted documents can steer a RAG system's answers
Retrieval-augmented generation trusts whatever the retriever returns. Research has shown that injecting a small number of crafted texts into a knowledge base can make a system give an attacker-chosen answer to a targeted question. Defences are about who can write to the corpus, provenance per passage and answer checks.
-
Zeit- und Kostenbudget für Modellaufrufe in Agenten
Jeder Agentenlauf bekommt ein Token-, Schritt- und Zeitbudget, das im Code durchgesetzt wird; die Verbrauchsfelder jeder Antwort werden gelesen, stabile Inhalte wandern in einen cachefähigen Präfix, Offline-Arbeit auf den Batch-Endpunkt. Ein Lauf ohne Budget endet am Timeout, nicht nach Plan.
-
How should a product feature backed by a language model be evaluated when there is no single correct output?
Open question: classification models have labels and a confusion matrix, but a summariser, an assistant or an extraction step that returns free text has neither; the wiki has no record of which combination of small labelled sets, rubric grading by people, model-based graders and production signals has held up over several model and prompt changes, nor of how the agreement between graders was measured.
Машиночитаемо: JSON