Newer Models, Same Advantage

Summary

DharmaOCR, a Brazilian Portuguese-focused OCR model, is evaluated against newer Mistral OCR4 and Unlimited-OCR systems on a Portuguese-only benchmark. The article attributes its performance to a two-stage training pipeline: supervised fine-tuning on varied Portuguese documents, followed by Direct Preference Optimization (DPO) using comparative output preferences. Fine-tuning concentrates model capacity on Brazilian Portuguese vocabulary, syntax, and document structures, while DPO targets repetitive or incoherent generation, reducing inference time and cost and improving stability according to the article. DharmaOCR scored 0.925, compared with 0.798 for Mistral OCR4 and 0.7587 for Unlimited-OCR, while examples from handwritten ENEM essays show the competitors misreading Brazilian names and phrases that DharmaOCR handled correctly. The conclusion is that newer architectures can raise absolute performance, but directing finite resources at one domain can preserve a measurable specialist advantage.

Takeaways

1. 1

   On the Portuguese-focused benchmark, DharmaOCR scored 0.925, versus 0.798 for Mistral OCR4 and 0.7587 for Unlimited-OCR.

2. 2

   Supervised fine-tuning builds Brazilian Portuguese domain competency, while DPO suppresses repetitive or incoherent outputs and improves stability, inference time, and cost.

3. 3

   The article argues that newer architectures can increase a model's performance ceiling, but specialization can direct finite parameters and training resources more effectively toward a fixed domain.