confabulations

Well here's some: Confabulation/Hallucination - https://github.com/lechmazur/confabulations Failure to read context - https://georggrab.net/content/opus46retrieval.html Deleting tests to make them pass - https://www.linkedin.com/posts/jasongorman_and-after-it-did-... Going rogue and deleting data - https://x.com/jasonlk/status/1946069562723897802 Agent security nightmares because they are not in fact intelligent assistants - https://x.com/theonejvo/status/2015401219746128322 Failure to read or generate structured data - https://support.google.com/gemini/thread/390981629/llm-ignor... There are many, many examples, mostly caused by people thinking LLMs are intelligent and reasoning and giving them too much power (e.g. treating them as agents, not text generators). I'm sure they're all fixed in whatever new version came out this week though.

https://www.swebench.com https://swe-rebench.com https://livebench.ai/#/ https://eqbench.com/# https://contextarena.ai/?needles=8 https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com... https://artificialanalysis.ai/leaderboards/models https://gorilla.cs.berkeley.edu/leaderboard.html https://github.com/lechmazur/confabulations https://dubesor.de/benchtable https://help.kagi.com/kagi/ai/llm-benchmark.html https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard

GPT-5 set a new record on my Confabulations on Provided Texts benchmark: https://github.com/lechmazur/confabulations/

I benchmarked it on four of my benchmarks so far. Got first place in two of them: https://github.com/lechmazur/confabulations https://github.com/lechmazur/nyt-connections https://github.com/lechmazur/generalization https://github.com/lechmazur/writing

I do wonder about the role of test time compute in the blog post in terms of document understanding. A non reasoning output (or low test time compute setting) might easily misinterpret the text, but reasoning models can second guess, consider multiple objectives in turn, and can right the ship. I note that Gemini 2.5 has one of the lowest confabulation/hallucination rates according to this benchmark [1], so am surprised by the results in the blog. Also, I have found link hallucination and output quality improve when you restrict searches to, for example, only pubmed sources, and to provide the source link directly into the text (as opposed to Gemini deep research usual method for citation). One reason, I think, is that unrestricted search will get the paper, the related blog posts and press releases, weight them as equal (and independent!

If anyone is interested in a larger sample size comparing how often LLMs confabulate answers based on provided texts, I have a benchmark at https://github.com/lechmazur/confabulations/. It's always interesting to test new models with it because the results can be unintuitive compared to those from my other benchmarks.

One of the most interesting things about that model is its excellent score on the RAG confabulations (hallucination) leaderboard. It’s the 3rd best model overall, beating all OpenAI models, for example. I wonder what Alibaba did to achieve that. https://github.com/lechmazur/confabulations

I ran three more of my independent benchmarks: - Improves upon GPT-4o's score on the Short Story Creative Writing Benchmark, but Claude Sonnets and DeepSeek R1 score higher. (https://github.com/lechmazur/writing/) - Improves upon GPT-4o's score on the Confabulations/Hallucinations on Provided Documents Benchmark, nearly matching Gemini 1.5 Pro (Sept) as the best-performing non-reasoning model. (https://github.com/lechmazur/confabulations) - Improves upon GPT-4o's score on the Thematic Generalization Benchmark, however, it doesn't match the scores of Claude 3.7 Sonnet or Gemini 2.0 Pro Exp. (https://github.com/lechmazur/generalization) I should have the results from the multi-agent collaboration, strategy, and deception benchmarks within a couple of days. (https://github.com/lechmazur/elimination_game/, https://github.com/lechmazur/step_ga

I have a set of independent benchmarks and most also show a difference between reasoning and non-reasoning models: LLM Confabulation (Hallucination): https://github.com/lechmazur/confabulations/ LLM Step Game: https://github.com/lechmazur/step_game LLM Thematic Generalization Benchmark: https://github.com/lechmazur/generalization LLM Creative Story-Writing Benchmark: https://github.com/lechmazur/writing Extended NYT Connections LLM Benchmark: https://github.com/lechmazur/nyt-connections/ and a couple more that I haven't updated very recently.

The problem is confabulations. In my benchmark (https://github.com/lechmazur/confabulations/), you see models produce non-existent answers in response to misleading questions that are based on provided text documents. This can be addressed.

You shouldn't use the rate as an indicator. They did something similar to what I did on my hallucinations benchmark (https://github.com/lechmazur/confabulations/), only using questions where at least one model made a mistake. I added this note: "The benchmark includes questions where at least one LLM confabulated, in order to minimize the number of questions requiring human assessment. Because of this, and since the questions are intentionally adversarial, the absolute percentage should not be used to infer that LLMs frequently confabulate. This leaderboard does not reflect a "typical" hallucination rate." > instead of teaching the model how to look for answers from an external source (e.g. RAG) My benchmark specifically focuses on the RAG use case. Even with provided texts, current models still hallucinate.

Confabulations are decreasing with newer models. I tested confabulations based on provided documents (relevant for RAG) here: https://github.com/lechmazur/confabulations/. Note the significant difference between GPT-4 Turbo and GPT-4o.