OmniDocBench

This would be more helpful if it included DeepSeek-OCR, PaddleOCR-VL and MinerU 2.5. In general, I've found that OmniDocBench is a reliable benchmark, perhaps surprisingly because it is made by the same team as MinerU. They updated their benchmark table recently: https://github.com/opendatalab/OmniDocBench#end-to-end-evalu.... There are some other models that score above DeepSeek-OCR as well that I'm not as familiar with.

For everyone wondering how good this and other benchmarks are: - the OmniAI benchmark is bad - Instead check OmniDocBench[1] out - Mistral OCR is far far behind most Open Source OCR models and even further behind then Gemini - End to End OCR is still extremely tricky - composed pipelines work better (layout detection -> reading order -> OCR every element) - complex table parsing is still extremely difficult [1]: https://github.com/opendatalab/OmniDocBench

Have you ran your pipeline against an open benchmark like https://github.com/opendatalab/OmniDocBench?

It would be nice to see how it performs on this benchmark: https://github.com/opendatalab/OmniDocBench

do you know where they may have released benchmarks, or like how it compares to https://github.com/opendatalab/OmniDocBench. If its on par that would be pretty insane given its so small.