Nemotron vs LightOnOCR vs Docling on Hard Invoices

Résumé

Pixel’s announcement examines a hard-invoice test comparing Nemotron 3.5 Parse, LightOnOCR 3, and Docling on scanned GST and Hindi invoices. Across 689 field and table checks, LightOnOCR performed best, while Nemotron missed a column and total and Docling struggled with Devanagari—insights that matter for reliable invoice extraction.

DEV Community

Nemotron 3.5 Parse vs LightOnOCR 3 vs Docling on hard invoices: what the numbers mean

NVIDIA released a small document parser in September, Nemotron 3.5 Parse: 631 million parameters, free to download, and top of its own ParseBench table. A few weeks later Laxmi Kant of KGP Talkie Hindi put it through a test I liked: two deliberately hard scanned invoices, one GST invoice and one in Hindi, run through Nemotron, LightOnOCR 3 and Docling on a MacBook Pro M5 Max. The result was not what NVIDIA's table would predict. LightOnOCR 3 read almost everything. Nemotron dropped a column and the grand total. Docling turned Hindi into nonsense. The reel is under three minutes, so it shows the scores and not much of the why. I spent an evening on the why: the model cards, Docling's source, and a small reproduction of the Hindi failure on my own Mac. This post covers what the numbers mean and what I would do if I were building invoice extraction today. Numbers from the KGP Talkie Hindi reel. I did not rerun the three models. The test in the reel Documents: a two-page GST invoice and a Hindi invoice, printed, scanned, tilted, noisy and folded. What was checked: 86 key fields (invoice numbers, tax IDs, totals, bank details) plus every table cell, 689 checks in all. The strict rule: a cell only counts if the value lands in the right row. Reading 12,510.36 correctly but putting it on the carton row instead of the HDPE jar row is a miss. Hardware: MacBook Pro M5 Max, every page timed. That row rule matters more than it sounds. Most OCR scores count characters. An accounts team cares whether the amount sits next to the right line item, because that is what gets posted to the ledger. The three results, and what went wrong in each Docling: fast, and blind to Devanagari Docling took about 12 seconds a page and scored 93.7% on the GST invoice and 81.7% on the Hindi one. The reel shows one line that explains most of the gap: the printed text निर्माण / Mfg came out as fARFUT / Mfg. The English half survived. The Hindi half was read as if it were Latin letters. That is not really Docling's layout model failing. Docling hands the text recognition to a separate OCR engine, and the default choice depends on your machine. Here is the selection order from OcrAutoOptions in Docling's current source: On a Mac the first match is OcrMac, which is Apple's Vision framework. So I asked Vision on my own machine (macOS 15.7) which languages its accurate recognizer supports: langs.swift import Vision
let r = VNRecognizeTextRequest()
r.recognitionLevel = .accurate
print(try r.supportedRecognitionLanguages().joined(separator: " "))
en-US fr-FR it-IT de-DE es-ES pt-BR zh-Hans zh-Hant yue-Hans yue-Hant ko-KR ja-JP ru-RU uk-UA th-TH vi-VT ar-SA ars-SA
No Hindi. To see what that does to an invoice, I rendered three lines in a Devanagari font, including the same grand total that appears in the reel, and ran Vision on them two ways: with Docling's OcrMac default languages, and with automatic language detection. The test image: Kohinoor Devanagari, black on white, no noise at all. output langs=en-US,es-ES,fr-FR,de-DE
Mg
del T: 76,28,866.00
langs=auto
Mg
엽의 시 생6,28,866.00
This is a clean image, with no tilt, no fold and no scanner noise, and it still fails badly. The whole amount-in-words line is gone. Worse, the rupee sign was read as a 7, so ₹6,28,866.00 became 76,28,866.00. That is a number that parses fine, passes a type check and is wrong by 70 lakh. It is the kind of error that reaches a ledger. The fix is to choose an engine that has a Devanagari model instead of taking the default. EasyOCR has one. I wrapped the same image in a PDF and ran this with Docling 2.150.1: docling_hindi.py from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import EasyOcrOptions, OcrMode, PdfPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption

opts = PdfPipelineOptions(
do_ocr=True,
# Scans have no text layer, so OCR the whole page. "hi" pulls in EasyOCR's Devanagari model.
...
Nemotron vs LightOnOCR vs Docling on Hard Invoices
Powered by Readybot.io

Les dernières actus de Pixel

Incident Response: Look Beyond the Red Alert

Pixel’s DEV Community announcement explores how teams can fixate on one failing service while overlooking the real cause of an outage, such as a full …

Sach: AI Fact-Checks WhatsApp Forwards in India

Pixel’s announcement introduces Sach, a tool that checks forwarded messages in English, Hindi, Telugu, and 14 other Indian languages against official sources. It gives cited …

Java 25 Removes SecurityManager: What to Fix

Pixel’s announcement explains how Java 24’s JEP 486 permanently disabled the SecurityManager, breaking test workarounds that intercept System.exit. It recommends returning exit codes or throwing …