FABLES

HN post re: FABLES: https://news.ycombinator.com/item?id=39982362 FABLES/booklist.md: https://github.com/mungg/FABLES/blob/main/booklist.md /gscholar_related? FABLES: https://scholar.google.com/scholar?q=related:Y-Hx-kplbEUJ:sc... /gscholar_citations? BoookScore: https://scholar.google.com/scholar?cites=1796862036168524911... ... From that one day awhile ago: https://news.ycombinator.com/item?id=38347868#38354679 : > "LLMs cannot find reasoning errors, but can correct them" [ https://arxiv.org/abs/2311.08516 ] https://news.ycombinator.com/item?id=38353285

As far as I can tell, this study was almost entirely about fiction: https://github.com/mungg/FABLES/blob/main/booklist.md lists 26 books, only one of which is classified as non-fiction. I would imagine that summaries of non-fiction books are evaluated quite differently from summaries of fiction. I've been trying to figure out what prompts they used. The https://github.com/mungg/FABLES GitHub repo says this: Summary -- (str) Entire book summarized by one of five models: Mixtral, GPT-3.5-Turbo, GPT-4, GPT-4-Turbo, and Claude-3-Opus, using the hierarchical merging method described in Chang et al.. With a link to https://arxiv.org/pdf/2310.00785.pdf - which then links to another GitHub repository, https://github.com/lilakk/BooookScore which has a bunch of prompts in https://github.com/lilakk/BooookScore/tree/main/prompts