ocr-benchmark

If you're wondering how they prompt the models: "Perform OCR on this image. Return only the text found in the image as a single continuous string without any newlines, additional text, or commentary. Separate words with single spaces. For any truncated, partially visible, or occluded text, include only the visible portions without attempting to complete or guess the full text. If no text is present, return empty double quotes." Found in: https://github.com/video-db/ocr-benchmark/blob/main/prompts....

A new benchmark study evaluates Vision-Language Models (Claude-3, Gemini-1.5, GPT-4o) against traditional OCR tools (EasyOCR, RapidOCR) for extracting text from videos. The findings show VLMs outperforming OCR in many cases but also highlight challenges like hallucinated text and handling occluded/stylized fonts. The dataset (1,477 manually annotated frames) and benchmarking framework are publicly available to encourage further research. Paper: https://arxiv.org/abs/2502.06445 Dataset & Repo: https://github.com/video-db/ocr-benchmark Would love to hear thoughts from the community on the future of VLMs in OCR.