Vision LLM and OCR for KYC/AML: Document Intelligence

The arrival of multimodal Large Language Models has had a significant impact on the document management processes that shape the operational workflows of many companies. In enterprise contexts the hardest challenge is not extracting the text on a scanned document, but turning it into structured, reliable, verifiable data that the corporate information system can actually use.

One particularly critical use case, which we have faced repeatedly, concerns KYC, KYB and AML obligations. These are contexts in which simply extracting a name, a date or a percentage is not enough to complete the process: you have to understand what those values mean, which entity they refer to, which document they come from, and how reliably they can be used to move the case forward.

For years the term OCR was treated as a synonym for document automation. Over the past few years Vision LLMs and, more generally, Document Intelligence systems have shifted the focus from pure text transcription to understanding the document as a whole: layout, relationships between fields, tables, signatures, stamps, checkboxes, semantic consistency and evidence quality.

It is important, however, not to fall into a misleading simplification: Vision LLMs are not a replacement technology for classic OCR. In the real projects we have built over the last two years the most effective scenario has almost always been a hybrid architecture, in which traditional OCR, document parsers, multimodal models, deterministic rules and human review all collaborate inside the same workflow.

This article offers a pragmatic reading of the subject and a framework you can use to design document systems that are genuinely reliable in production.

A readable document is not yet reliable data

A system can correctly recognise the string 37% and still not have reached its functional goal. In a corporate document that percentage could indicate a direct holding, an indirect holding, a historical shareholding, a class of shares, or a value referring to an entity other than the expected one.

The difference between simple reading and true document understanding lies precisely in that interpretation. An enterprise workflow must not only answer the question “what text appears on the page?”, but also more semantically relevant questions:

• what does this value mean;
• which entity does it refer to;
• where in the document does it appear;
• how reliable is the extraction;
• which checks have already been carried out;
• whether the value can be used automatically or requires human review.

The quality of automation in a regulated process is measured by its ability to produce structured, verifiable evidence rather than by the quality of the OCR.

What changes from OCR to Vision LLMs

OCR was born to recognise the individual characters of a text. When conditions are good and the document is relatively standardised it is an extraordinarily useful technology: fast, inexpensive, deterministic and, in most cases, incapable of inventing information that is not there.

A Vision Language Model, by contrast, combines visual and linguistic information. It does not simply extract the characters contained in the text; it tries to interpret several syntactic and semantic elements at the same time:

• page structure;
• spatial relationships between blocks and fields;
• semantic context;
• tables;
• charts;
• non-textual elements such as signatures, stamps and checkboxes.

LLMs natively extract and process information about the position of tokens — that is, of textual elements: words, syllables, acronyms — about their relationships with the other elements on the page, and about the type of document in which the text appears.

Text extraction does not happen by simply classifying the marks in the image: it is the result of a generative inferential process, characteristic of LLMs.

The greater power and versatility of this technology lies precisely in that nature, but the flip side is that extracted information may in some cases appear to be invented, through the phenomena known as hallucinations — also characteristic of LLMs and, above all, impossible to prevent entirely.

OCR: an excellent reader, but not always a sufficient interpreter

OCR continues to excel wherever the problem is mainly one of pure text recognition:

• repetitive forms;
• very stable layouts;
• high volumes with cost and latency constraints;
• offline environments or strong data residency requirements;
• processes that need predictable output.

The limit emerges when the document leaves those rails: degraded scans, complex tables, ambiguous fields, variable layouts, semantic relationships that are not expressed uniformly.

Vision LLMs: structure and context beyond text

Vision LLMs provide a capability that pure OCR does not natively possess: understanding the document as a structured object.

A Vision LLM can recognise that a given string is a person’s name, that a date is an incorporation date, that a percentage expresses a shareholding, and that two blocks of text belong to the same logical section.

That is why their contribution becomes valuable in processes where structured data must be extracted from non-standardised documents.

OCR and Vision LLMs are not opposites

The right question is not: “Which technology is better?”

The right question is: “Which technology makes sense for this type of document, in this language, at this cost, with this level of risk and this need for auditability?”

Why KYC, KYB and AML are the perfect proving ground

KYC and KYB workflows are probably the harshest test for a Document Intelligence pipeline, because they present almost every possible difficulty within a single process.

A real KYC/AML scenario has to handle:

• identity documents;
• proofs of address;
• registration certificates;
• register extracts;
• articles of association;
• shareholder registers;
• beneficial ownership declarations;
• powers of attorney;
• heterogeneous corporate deeds;
• sanctions and PEP screening;
• checks on control and ownership chains.

To this, three further complications are added — and within the framework of anti-money laundering obligations they weigh as much as the rules themselves.

1. Documents differ from country to country

The same information can appear in radically different formats depending on the jurisdiction. A French company extract, a German Handelsregister, an Anglo-Saxon certificate of incorporation and an Asian business registration document do not share the same structure, the same terminology, or even the same graphic conventions.

Europe has taken important steps towards interconnecting business registers — the BRIS system lets you search for a company across all Member States from a single access point — but access to data is not the same as full documentary harmonisation. In practice, enterprise workflows continue to deal with national documents, local languages and deeply heterogeneous formats.

2. The relevant information is relational

In KYC and KYB it is not enough to extract independent fields. Relationships have to be reconstructed:

• who controls whom;
• who is a director and who is a shareholder;
• which stakes multiply along an ownership chain;
• who does or does not fall within the UBO threshold;
• whether the data in several documents are consistent with each other.

Reading characters alone is not sufficient here: context has to be understood.

3. Auditability is part of the result

In an AML process the extracted value is useless if it cannot be traced clearly back to its source. An operator must be able to check quickly:

• which document the value comes from;
• which page it is on;
• whether the crop or bounding box is correct;
• what confidence was assigned;
• which control rules were applied;
• why the system decided to accept, reject or send the case to review.

In other words, in regulated processes the useful output is the value plus the proof.

The real multilingual problem is not the number of languages supported

Many vendors claim coverage of dozens or even hundreds of languages. That is useful information, but not enough to assess the technology.

The critical point is how the system behaves on the real documents of the jurisdictions that make up the company’s portfolio.

This distinction is essential for a multinational. An average English-language benchmark does not automatically predict behaviour on:

• Arabic;
• Chinese;
• Japanese;
• Korean;
• Hindi and other Indic scripts;
• mixed documents with Latin and non-Latin text;
• real degraded scans;
• documents with unusual fonts or low-quality printing.

Above all, it says nothing about the ability to handle documents structured according to local practice and conventions.

The evidence that emerged in 2026 from benchmarks dedicated to under-represented scripts shows exactly this: models that look very strong in mainstream contexts can degrade dramatically on real documents in less frequent languages and writing systems.

For an international organisation the lesson is clear: do not evaluate the system on ideal documents, but on the real documentary long tail.

OCR or Vision LLM? That is the wrong question

In enterprise workflows the more useful question is not whether to choose OCR or a Vision LLM, but how to design the routing.

A well-designed architecture cannot rely on a universal model that does everything: it requires analysing, for each document class, the most efficient and most reliable processing path.

A simplified scheme drawn from our own experience looks like this:

1. classify the document;
2. detect format, language and input quality;
3. choose the most suitable pipeline;
4. extract data against a schema;
5. validate the results;
6. compute confidence and risk;
7. decide whether to proceed automatically or involve an operator.

In practice this means, for example, that:

• a native PDF can be handled with direct parsing;
• a very stable form can follow a classic OCR pipeline;
• a known table can be read with a deterministic parser;
• an unusual or poorly standardised document can be routed to a Vision LLM;
• a case with low confidence or inconsistencies can be sent to human review.

Seen this way, the real differentiator is not the individual model but the orchestration.

The architecture that works in production is hybrid

Research and our production experience converge on one point: the most effective systems tend to be hybrid. A 2025 study on repetitive identity documents shows that pairing an OCR engine with an LLM reaches an F1 of 1.0 with 0.97 seconds of latency on structured documents, while the purely multimodal approach remains far slower for the same result.

A Document Intelligence architecture for KYC/AML can be described as a sequence of specialised components:

1. File ingestion and validation
2. Document classification
3. Routing by format, language and quality
4. Parsing with the appropriate technology
   • native parser;
   • OCR;
   • Vision LLM;
   • specialised parser.
5. Schema-driven extraction
6. Deterministic validation
7. Cross-checks against external sources or other documents
8. Confidence and risk scoring
9. Straight-through processing or human review
10. Audit trail persistence

This approach offers several concrete advantages.

It maximises the cost-to-reliability ratio

Traditional OCR is often unbeatable on throughput, marginal cost and predictability. Vision LLMs become valuable when documentary and semantic variability makes template-based rules alone ineffective or too expensive.

Always using the most sophisticated model is not necessarily the best choice. At enterprise volumes it makes far more sense to use the most expensive model only where it genuinely adds value.

It reduces template fragility

Systems based solely on templates can be extremely effective as long as the document stays stable, but they become fragile the moment the layout varies. Vision LLMs can be the component that absorbs variability, leaving deterministic rules in control of the most sensitive steps.

It makes the long tail manageable

The hardest problem in document workflows is the so-called long tail: rare documents, imperfect scans, unexpected formats, infrequent language combinations, incomplete cases. A hybrid architecture lets you handle those cases without forcing the whole pipeline to work the same way for every document.

Read, Understand, Prove: a framework for designing Document Intelligence

To avoid the excess of hype it helps to reduce the problem to three very simple verbs.

1. Read

The system must read the document and recognise its fundamental elements:

• text;
• numbers;
• dates;
• tables;
• signatures;
• stamps;
• checkboxes;
• barcodes or QR codes;
• the basic structure of the page.

Here the most effective technologies are OCR and document parsers.

2. Understand

The system must understand the meaning of what it has read:

• what the value represents;
• which entity it refers to;
• whether the date is one of issue, expiry or incorporation;
• whether a percentage indicates ownership, control or something else;
• whether a person is a director, a shareholder or a legal representative;
• whether several documents tell a consistent story.

This is where Vision LLMs contribute most.

3. Prove

Finally, the system must turn the extraction into verifiable evidence:

• source document;
• page;
• bounding box or crop;
• confidence;
• validation rules applied;
• human review, if any;
• model and workflow version.

This is not a theoretical requirement: the latest generation of engines already returns a structured representation of the document, with each block localised by a bounding box, classified by type and accompanied by a per-page confidence score.

That third layer is what makes a system genuinely usable in a regulated context. The output expected from an enterprise application should not simply be:

• beneficial_owner = Mario Rossi
• ownership = 37%

but should carry indications such as:

• which document the value came from;
• where on the page it was found;
• with what confidence it was extracted;
• whether it is consistent with other documents;
• whether the case can proceed without human intervention.

When designing a document pipeline, one practical question therefore becomes essential: what share of the output is backed by evidence that can easily be reconstructed?

A document is also untrusted input

One element not to overlook is that in KYC workflows the document comes from outside. By definition, it must be treated as untrusted input.

That means every file should be treated as a possible vector of error, ambiguity or attack, as well as a source of data.

In the world of generative models the most discussed risks are:

• hallucinations;
• errors on proper names, numbers and dates;
• non-determinism;
• structural interpretations that are plausible but wrong;
• indirect prompt injection carried through external content.

That last point is not theoretical: prompt injection is the first entry in OWASP’s ranking of risks for LLM applications, and the indirect variant is precisely the one that arrives through external content such as files and documents. For this reason it is worth setting a simple principle:

The content of a document must be treated as data, not as instructions.

And a second, even more operational principle:

A component authorised to read a document must not automatically acquire the authority to act on corporate systems or sensitive data stores.

From these principles follow a number of security-hardening architectural choices:

• separate extraction from action;
• limit the privileges of components that process documents;
• use structured, validated output;
• introduce deterministic checks before any sensitive automation;
• require human approval for high-impact steps.

In regulated workflows, security must be an integral part of the pipeline design.

Human in the loop does not mean going back to manual work

One of the most frequent objections is this: if an operator has to check the result anyway, what is the advantage?

The answer is that human in the loop does not mean checking everything. It means checking only what genuinely deserves attention.

Automation creates value when it can reliably distinguish three categories:

1. simple, sufficiently certain cases, which can proceed automatically;
2. uncertain cases, which require a quick check;
3. anomalous or high-risk cases, which require an in-depth review.

Thresholds should depend on the risk carried by the data as much as on the model’s technical confidence. A postcode read with 92% confidence does not have the same impact as a beneficial owner or a shareholding that determines UBO classification.

The design maturity of a solution is measured by exactly these elements: the definition of thresholds, exceptions, escalation and review procedures that reduce manual work without weakening control.

How to evaluate a Vision LLM/OCR pipeline

One of the most common mistakes is to settle for generic accuracy metrics, or for public benchmarks read superficially. Production instead calls for specific metrics that reflect the real process.

Some genuinely useful KPIs

Examples of appropriate performance indicators are:

• Field precision / recall: how correct the individual fields are.
• Document-type accuracy: how well the system classifies the document.
• Schema completeness: how many mandatory fields are actually returned.
• Cross-document consistency: how consistent the data are across sources.
• Human-review rate: how many cases require human intervention.
• Straight-through processing rate: how many cases run end-to-end automatically.
• Accuracy by language: how quality varies by language or script.
• Accuracy by jurisdiction: how quality varies by country.
• Accuracy by document type: where the long tail of errors concentrates.
• Catastrophic error rate: how often the system produces serious errors.
• Cost per processed case: not just cost per page or per token, but real cost per case.
• P95 latency: operational performance on real cases.
• Model/version drift: stability over time after model or workflow updates.

One metric that deserves attention: evidence

In many regulated contexts a pipeline should be assessed not only for what it extracts, but for how well it can prove where the extracted information came from.

Alongside the classic metrics it is therefore useful to introduce a criterion such as evidence coverage: the share of output for which the system retains proof that is easy to reconstruct and verify.

Public benchmarks should be read with care

Benchmarks such as OCRBench v2, and the more recent studies on how models behave on real documents and under-represented scripts, are very useful for orientation, but they do not replace the elementary rule that the system must be tested on your own real documents.

The 2026 study on Devanagari is a good example of how wide the gap can be: the ten systems compared — from classic EasyOCR to frontier multimodal models — all sit between 91 and 98 chrF++ points on clean synthetic text, yet on 300 real scans nine out of ten collapse and the field spreads across a 76-point range. The same work notes that strong English OCR ability does not predict behaviour on Indic scripts.

A global average can therefore hide precisely the errors that weigh most in a specific business context.

AI Act, AML and governance: no shortcuts

When AI is applied to KYC it is easy to fall into excessive simplifications, especially on the regulatory side.

The first is to assume that any use of AI in KYC automatically amounts to a high-risk system. In reality the classification depends on the system’s intended purpose and on the concrete role it plays in the process: Annex III of the AI Act lists eight areas, and a document extraction engine does not fall into them merely because it uses a model.

The second simplification is to imagine that the only issue is future regulatory compliance — which is not that far off, given that enforcement of the rules on Annex III high-risk systems starts on 2 December 2027. In reality many organisations already have very concrete needs for:

• traceability;
• model versioning;
• documentation of rules;
• change control;
• human oversight;
• operational explainability.

This is not only an internal matter. ECB Banking Supervision, reviewing the use of AI in European banks, observed that some institutions lack full transparency over the internal workings of certain models: exactly the point where a well-designed document pipeline can make the difference.

For this reason it is more useful to frame the subject in terms of system governance than in slogans. A well-designed architecture should make it possible to know:

• what the system did;
• with which model versions;
• on the basis of which evidence;
• under which validation rules;
• with what human intervention, if any.

This is the minimum basis for using AI effectively inside regulated document workflows.

What really changes for a multinational

The problem for multinational organisations is not that document volumes are much higher than in smaller organisations, but that variability is much greater.

Volume is handled by scaling processing capacity; variability, which follows directly from the multinational character of these organisations, is an application and functional problem instead.

In a multinational, what grows is:

• the number of document types;
• national exceptions;
• languages and scripts;
• incomplete or degraded cases;
• compliance and data residency constraints;
• the need to standardise output from deeply heterogeneous input.

In that context the answer is not to read each individual page “slightly better”, but to reduce the dependency on an excessive number of country-specific parsers, rules and templates. This is where multimodal models show their full value.

Put differently: the real benefit appears when the system can absorb more variability without losing control. Multimodal models are used to handle the mess of the real world; rules, validation and review keep reliability, auditability and governance in place.

Conclusion: the model at the top of the benchmark is not the best solution

Moving from OCR to Document Intelligence is not about replacing a recognition engine with a bigger model.

It is about designing a pipeline that can:

• decide how to treat each document;
• read text and structure;
• understand what the data mean;
• produce verifiable evidence;
• trigger deterministic checks;
• route uncertain cases to human review;
• keep cost, latency and governance within production-compatible limits.

For this reason, when designing complex KYC/AML workflows, there is little value in assessing which model currently leads a benchmark.

The more useful question a designer should ask is a different one:

Can the system produce correct, verifiable and governable data on the real documents the organisation works with every day?

If the answer is yes, then the proposed solution can properly be called Document Intelligence designed to work in production.

This is the ground our artificial intelligence services are built on: starting from the real documents, the operational volumes and the process control requirements, before the model.