doctr

checkout https://github.com/mindee/doctr or https://github.com/VikParuchuri/surya for something practical multimodal llm would of course blow it all out the water, so some llama3-like model is probably SOTA in terms of what you can run yourself. something like https://huggingface.co/blog/idefics2

https://github.com/mindee/doctr/issues/1049 https://github.com/JaidedAI/EasyOCR#whats-coming-next Happy to see OCR is advancing lately, but I really need HWR. I am looking for something this polished and reliable for handwriting, does anyone have any pointers? I want to integrate it in a workflow with my eink tablet I take notes on. A few years ago, I tried various models, but they performed poorly (around 80% accuracy) on my handwriting, which I can read almost 90% of the time.

Tesseract alone is widely known to be "meh" at this point. If you look at RAG frameworks as one example they'll typically use/support a variety of implementations. Tesseract is almost always supported but it's rarely ideal with projects like Unstructured[0] and DocTR[1] being preferred. By leveraging more-or-less SOTA vision models[2][3] they embarrass Tesseract. I haven't compared them to the Apple Vision framework but they're absolutely better than Tesseract and potentially even Apple Vision. There are also various approaches to use these in conjunction but that gets involved. [0] - https://github.com/Unstructured-IO/unstructured-inference [1] - https://github.com/mindee/doctr [2] - https://github.com/mindee/doctr#models-architectures [3] - https://github.com/Unstructured-IO/unstructured-inference#mo...

Yup! But I'm still exploring options. (any recommendations would be welcomed!) Here are some candidates I'm considering: - https://github.com/mindee/doctr - https://github.com/open-mmlab/mmocr - https://github.com/PaddlePaddle/PaddleOCR (honestly I don't know Mandarin so I'm a bit stuck) - https://github.com/clovaai/donut -- While it's primarily an "OCR-free document understanding transformer," I think it's worth experimenting with. Think I can sort this out by letting the LLM reason through it multiple times (although this will impact performance) - yesterday got a suggestion to consider https://github.com/kakaobrain/pororo -- don't think development is still active but the results are pretty great on Korean text

EasyOCR is a popular project if you are in an environment where you can use run Python and PyTorch (https://github.com/JaidedAI/EasyOCR). Other open source projects of note are PaddleOCR (https://github.com/PaddlePaddle/PaddleOCR) and docTR (https://github.com/mindee/doctr).

Last I checked I saw a grocery bill example using https://github.com/mindee/doctr and was fairly accurate. Bear in mind that was last year, hopefully it got even better or there are other libraries

There's also DocTR which can do text detection and extraction out of the box. It's command line driven but can display the detected text as an overlay of the document. https://github.com/mindee/doctr

If you want to OCR a document image, modern versions of Tesseract can work well. If you last used it a few years ago, the recognition has improved since due to a new text recognition algorithm that uses modern (deep learning) techniques. Browser demo using a modern version: https://robertknight.github.io/tesseract-wasm/. OCR processing typically consist of two major steps: detecting/locating words or lines of text on the page, and recognizing lines of text. Tesseract's text recognition uses modern methods, but the text detection phase is still based on classical methods involving a lot of heuristics, and you may need to experiment with various configuration variables to get the best results. As a result it can fail to detect text if you present it with something other than a reasonably clean document image. Doctr (https://github.com/minde

DocTR: https://github.com/mindee/doctr It also has TensorFlow.js version to run in-browser: https://blog.tensorflow.org/2022/06/ocr-in-browser-using-ten...