exploitgym

evaluation awareness is a (at this point) well-known phenomenon among LLMs. It seems the better they get, the more often they're able to guess whether they're in an evaluation environment. Clues usually exist, like being in a sandbox, having restricted network access, having no human interaction. Beyond that, exploitgym's prompts are pretty clearly CTF (capture the flag) style that make it obvious it's an evaluation or competition: one prompt [1] tells the LLM to read the file path "/secret", another prompt [2] talks about finding flags and where to write them out. [1]: https://github.com/sunblaze-ucb/exploitgym/blob/main/src/cyb... [2]: https://github.com/sunblaze-ucb/exploitgym/blob/main/src/cyb...

Interestingly if you look at the exploitgym repo (https://github.com/sunblaze-ucb/exploitgym/blob/e5ea7c233a4d...) the intended run mechanism is orchestrated by some python scripts which run agents against various prompts. The prompts themselves wouldn’t mention anything about exploitgym and there should be nothing steering the agent towards trying to find the answers out-of-band. So I don’t see how they could even run into this problem unless all they did was tell an agent to “check out and run exploitgym” Which, after hearing some personal anecdotes of how people work there, seems plausible

1. The benchmark is run with a python script - https://github.com/sunblaze-ucb/exploitgym using an agent harness. I suspect they used codex. So the model has access to the environment and could trivially inspect its own source code, which has lots of references to exploit gym and docs relating to it. 2. Hanlons Razor

Even if I get downvoted I will mention that I agree with you. > all with reduced cyber refusals for evaluation purposes — while being internally tested on a benchmark (opens in a new window) of cyber capabilities. OpenAI was testing their cyber[1] variant of their models with reduced safeguards and the prompt likely specified things related to exploits given that it was tackling problems from ExploitGym[2]. For those who don't know what ExlploitGym is, see the description on their Github page which is pasted below: > ExploitGym is a large-scale, realistic benchmark built from real-world vulnerabilities designed to evaluate AI agents' ability to develop exploits. From OpenAI's statement [3]: > We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cybe