surya

Thanks! Honestly? Initially very poorly. What I ended up doing was generating around a dozen versions of a screenshot in realtime (all with different combinations of thresholding, segmentation parameters, resolution scaling, and denoising) behind the scenes. Then it would fire Tesseract off on all of them in parallel threads and let them “vote” on the result. After I set that up, the accuracy improved significantly. If you're looking for an alternative rather than Tesseract - I'd actually recommend Surya. I've had a lot of success with it out of the box with doing OCR on comics. https://github.com/datalab-to/surya

I have tried a bunch of things. This is what worked best for me: Surya [0]. It can run fully local on your laptop. I also tried EasyOCR [1], which is also quite good. I haven't tried this myself, but I will look at Paddle [2] if the previous two don't float your boat. All of these are OSS, and you don't need to pay a dime to anyone. [0]: https://github.com/VikParuchuri/surya [1]: https://github.com/JaidedAI/EasyOCR [2]: https://github.com/PaddlePaddle/Paddle

pypdfium2 is a great choice and a solid piece of software! You might want to look into https://github.com/VikParuchuri/surya as an alternative to tesseract. Yes, it's associated with a commercial company, but as you long as you aren't a company with 5M in ARR or $5M in funding it's free to use.

I was just trying a bunch of models for OCR. I only have 4 GB of VRAM in my personal machine. My goal was to run an OCR model locally and extract text from scanned PDFs. Many models could not even be run. Among those that did run, thanks to Ollama, provided very poor experience. Like llava-llama3, phi3.5 vision, etc. What worked really well, but still not up to the mark- Surya [0]. It works perfectly on screenshots from true text PDFs, but not from scanned PDFs. Also has much better performance for English than Indian languages. [0]: https://github.com/VikParuchuri/surya

Docling is a great project, happy to see more people building in the space. Marker output will be higher quality than docling output across most doc types, especially with the --use_llm flag. A few specific things we do differently: - We have hybrid mode with gemini that merges tables across pages, improves quality on forms, etc. - we run an ordering model, so ordering is better for docs where the PDF orde ris bad - OCR is a lot better, we train our own model, surya - https://github.com/VikParuchuri/surya - References and links - Better equation conversion (soon including inline)

Surya is a great open source toolkit for table parsing, layout analysis and OCR: https://github.com/VikParuchuri/surya

Datalab | NYC | Full-time | Software Engineer and Head of Business Ops | $250k-$350k + 1.5-3% equity | https://www.datalab.to A significant % of useful data is locked away in tough-to-parse formats like PDFs. We build tools to extract it, like https://github.com/VikParuchuri/surya (15k Github stars), and https://github.com/VikParuchuri/marker (19k stars). We also run an inference API and product. We do meaningful research (we’ve trained several SoTA models), ship product, and contribute to open source. We’re hiring for 2 roles to help us scale: Senior fullstack software engineer - work across our open source repos, inference api, and frontend product - interact with our user community - you’ll need to be pragmatic, and embrace “boring” technology. Our stack is fastapi, pytorch, htmx, postgres, and redis. We deploy to render, a

I’ve used Surya (https://github.com/VikParuchuri/surya) before. It is very good (on par with Google Vision, potentially better layout analysis), but yours is a challenging use case. I wonder if it would be useful.

Hi, I'm the author of surya (https://github.com/VikParuchuri/surya) - working on improving speed and accuracy now. Happy to collaborate if you have specific page types it's not working on. For modern/clean documents it benchmarks very similarly to Google Cloud, but working on supporting older documents better now.

Surya https://github.com/VikParuchuri/surya and Marker https://github.com/VikParuchuri/marker

LLMWhisperer from Zipstack at https://llmwhisperer.unstract.com/ or https://github.com/VikParuchuri/surya will do a good job for you. LLMWhisperer has some nice tooling where they can fall back to OCR as well forcing text extraction from scanned documents as well as documents that have the text preserved as text.

checkout https://github.com/mindee/doctr or https://github.com/VikParuchuri/surya for something practical multimodal llm would of course blow it all out the water, so some llama3-like model is probably SOTA in terms of what you can run yourself. something like https://huggingface.co/blog/idefics2

Would this be able to incorporate the models from Surya — https://github.com/VikParuchuri/surya

This looks great! You might be interested in surya - https://github.com/VikParuchuri/surya (I'm the author). It does OCR (much more accurate than tesseract), layout analysis, and text detection. The OCR is slow on CPU (working on it), but faster than tesseract (CPU-only) on GPU. You could probably replace pymupdf, tesseract, and some layout heuristics with this. Happy to discuss more, feel free to email me (in profile).

Wow, this is promising. I tried on a few poorly scanned papers I've lying about. A few observations: 1. Pre-process PDF images to detect letters better? 2. Use LLMs to spell/grammar check and perhaps even auto-complete missing pieces? 3. Employ rich text to capture style (ex: lexical.dev)? Unsure if it is feasible to bundle it all up for web. See also: https://github.com/RajSolai/TextSnatcher / https://github.com/VikParuchuri/surya