llm-security

TLDR: With these vulnerabilities, we show the following is possible: - Remote control of chat LLMs - Persistent compromise across sessions - Spread injections to other LLMs - Compromising LLMs with tiny multi-stage payloads - Leaking/exfiltrating user data - Automated Social Engineering - Targeting code completion engines There is also a repo: https://github.com/greshake/llm-security and another site demonstrating the vulnerability against Bing as a real-world example: https://greshake.github.io/ These issues are not fixed or patched, and apply to most apps or integrations using LLMs. And there is currently no good way to protect against it.

I think that almost all use cases for LLMs that process untrusted inputs are unsafe. See https://greshake.github.io/ and https://github.com/greshake/llm-security for more information.

Answering to myself as I can't edit my response any more. I've investigated a bit more and the attack as described by Greshake[1] seems to be much more realistic to me than I initially thought. [1] https://github.com/greshake/llm-security

Output from the language model is also being injected into a script that is then executed: https://github.com/jjuliano/aifiles/blob/ef529fd6281eaf8d373... He argued below that he is not vulnerable to indirect prompt injection attacks (https://github.com/greshake/llm-security), but I think he is wrong.

If you had a PDF reader which allowed arbitrary code execution on opening a file, would you argue the same? You give arbitrary read/write to the LLM, right? So ransomware, causing network requests as side effects etc. could all be possible. Look at the paper to find more descriptions of what could go wrong: https://github.com/greshake/llm-security

Corresponding Github repository: https://github.com/greshake/lm-safety

In this paper, we showcase the potentially brutal consequences of connecting LLMs to applications (like search). We propose newly enabled attack vectors and techniques and discuss them: - Remote control of chat LLMs - Persistent compromise across sessions - Spread injections to other LLMs - Compromising LLMs with tiny multi-stage payloads - Leaking/exfiltrating user data - Automated Social Engineering - Targeting code completion engines We also provide proof-of-concept demonstrations on GitHub: https://github.com/greshake/lm-safety This paper should be a must-read for anyone that is building a business by integrating LLMs right now. TLDR: Giving an LLM any interface to the outside, like a search capability, has critical security implications. When prompt injection is used and delivered by adversaries instead of the user themself, bad thin