annotateai

Shell: GNU parallel, pdftotext Python: PyPdf2, PdfMiner.six, Grobid, PyMuPdf; pytesseract (C++) paperetl is built on grobid: https://github.com/neuml/paperetl annotateai: https://github.com/neuml/annotateai : > annotateai automatically annotates papers using Large Language Models (LLMs). While LLMs can summarize papers, search papers and build generative text about papers, this project focuses on providing human readers with context as they read. pdf.js-hypothes.is: https://github.com/hypothesis/pdf.js-hypothes.is: > This is a copy of Mozilla's PDF.js viewer with Hypothesis annotation tools added Hypothesis is built on the W3C Web Annotations spec. dokieli implements W3C Web Annotations and many other Linked Data Specs: https://github.com/dokieli/dokieli : > Implements versioning and has the notion of immutable resources. > Embedding da

Very interesting work! If biomedical research and paper analysis is of interest to you, I've been working on a set of open source projects that enable RAG over medical literature for a while. PaperAI: https://github.com/neuml/paperai PaperETL: https://github.com/neuml/paperetl There is also this tool that annotates papers inline. AnnotateAI: https://github.com/neuml/annotateai

Shameless self and friend plug, but the world of extractive summarization is to thank for this idea. We've always known that highlighting and citations are important to ground models - and people. https://github.com/Hellisotherpeople/CX_DB8 https://github.com/neuml/annotateai

Excellent project. As mentioned in another comment, I've put together an embeddings database using the arxiv dataset (https://huggingface.co/NeuML/txtai-arxiv) recently. For those interested in the literature search space, a couple other projects I've worked on that may be of interest. annotateai (https://github.com/neuml/annotateai) - Annotates papers with LLMs. Supports searching the arxiv database mentioned above. paperai (https://github.com/neuml/paperai) - Semantic search and workflows for medical/scientific papers. Built on txtai (https://github.com/neuml/txtai) paperetl (https://github.com/neuml/paperetl) - ETL processes for medical and scientific papers. Supports full PDF docs.