Two ways to industrialise document AI
Automating a regulated document flow can be done in two ways: build your own models, or deploy a vendor platform inside your own perimeter. Both can reach the same level of performance in a proof of concept. What separates them is the maintenance and evolution effort each implies — and who carries it.
Business transactions that few documents make simple
In banking and insurance — and in life insurance above all — business transactions rest on heterogeneous document bundles: subscription forms, mandates, identity documents, supporting evidence, statements, notarial deeds. Each exists in as many variants as there are distributors, products and regulatory vintages, and a single case brings together dozens of pages whose useful information is scattered rather than filed.
Validating such a case is not a matter of reading a form: it means cross-checking documents against one another, verifying completeness, controlling consistency and applying thousands of business and regulatory rules. That places two demands on the model. It must distribute its attention efficiently across pages, going to find a piece of information where it actually sits rather than treating each page in isolation. And it must apply those rules accurately enough for the output to be usable without rework.
It is that level of demand that makes the operating model the decisive question: a frozen model holds for a while, then falls behind as formats evolve.
Maintenance decides the cost, not development
A development quote describes the build. Yet across a system’s life the build is the smaller of the two expenses — and the gap widens further as soon as a learning system is involved.
Maintenance is 60–80% of a classical software life cycle, and more still for an AI system, whose model code is only a small fraction of the delivered whole.1,2,3
Annotation, MLOps, drift detection, human correction, orchestration, compliance, version upgrades, reversibility: they do not disappear, they change payer.
Cost of ownership, in multiples of initial development. The dark part is everything that comes after delivery.
The consequence comes down to one number: where classical software costs about three times its initial development in total, an AI system costs about six — the multiple to budget for is twice as high. Model code is only a small fraction of the deliverable; the rest — data, annotation, validation, drift monitoring — is operations, and operations decide the three-year cost.
Why maintenance never stops
In banking and insurance, incoming documents change continuously, for four entirely ordinary reasons. Each one degrades a frozen model, and correcting it requires a full cycle: detect the degradation, assemble and annotate a new batch, retrain, revalidate, redeploy.
New distributors, new partners
Each arrives with their own forms and their own layouts.
Regulatory change
It alters every form of a product at once.
Channel drift
Phone photographs, degraded scans, electronic signature stamps.
Portfolio transfers
An entire corpus enters the pipeline overnight.
Its costliest property is that it is silent: a drifting model raises no error, it simply returns steadily less accurate results. It has to be monitored and treated continuously.
The eight cost items to budget
Beyond initial development, these items exist in both approaches. The question is not whether they get paid for, but by whom.
- 1Initial annotation and re-annotation at every drift event
- 2MLOps platform: model registry, versioned datasets, continuous integration
- 3Drift detection, per-field confidence scoring, alerting
- 4Human correction interface and error recovery
- 5Business orchestration: ingestion, document splitting, routing, idempotence, audit trail
- 6Compliance: SIEM integration, secrets management, access control, security testing
- 7Version upgrades across the technical stack
- 8Reversibility, documentation, knowledge transfer, on-call
Where the load accumulates over thirty-six months
Maintenance is not paid once: it recurs, and it tracks the number of formats to cover rather than Moore’s law. That is what separates the two trajectories, far more than the gap between two initial quotes.
Cumulative load, base 100 = first year of a bespoke build.
Illustrative schematic, not a measurement. It renders two explicit assumptions: on the bespoke side, an annual maintenance load that recurs and grows with the number of formats to cover; on the platform side, the same load shared inside the licence, with a compute footprint that shrinks as models get more compact. Replace these assumptions with your own rates.
Bigger is not better
On a narrow document domain, model size is not the accuracy lever it is assumed to be: doubling the parameter count buys little, specialising buys seven times more. A document model is therefore judged on two axes that do not trade off against each other — what it weighs and how much it gets right. Every model quoted is evaluated on the same versioned test set, with the same metric.
- Why so light
- A model specialised on a document task does not need general knowledge, it needs accuracy on its domain: excess size costs compute and returns nothing
- Compute consumed
- ~100× less than a generic LLM
- What that buys you
- Execution inside your perimeter, on your own hardware, without depending on a third-party cloud
- Trajectory
- Footprint shrinks with each version
the gain from specialisation
- Against a generic LLM
- 5× more business accuracy
- Detecting the error
- Per-field confidence score and consistency checks across documents — average accuracy says nothing about what happens when the model is wrong
- Correcting in the flow
- Field-level rework without leaving the tool, and every correction sharpens the model on your domain
- Output conformity
- 0% unusable responses — one kind of error among others, but the kind that blocks a transaction
These two axes do not trade off against each other, and that is what makes specialisation pay: a compact model is cheaper to run while returning more accurate results on its domain, where the race for size buys both the wrong way round. Every model quoted is evaluated on the same versioned test set with the same metric — the comparison is direct, with nothing reconstructed.
The two operating models, line by line
Both approaches can share the essentials: execution inside your perimeter, no data egress, open-weight models. They differ in the operating model.
Across what base is model maintenance effort amortised?
| Criterion | Vendor platform, deployed at your site | Bespoke development |
|---|---|---|
| Process coverage at start | Pipeline proven in production, configurable from scoping onwards. | To be built process by process. |
| Human correction interface | Included — a product, not a screen to specify. | To be developed, tested and tuned. |
| Drift detection | Included and tooled, with per-field confidence scoring. | To be designed and costed. |
| Model update cadence | Continuous: every version starts from the best available open base and is included in the licence. | Frequency and price to be agreed. |
| R&D amortisation base | The vendor's entire customer base. | Your organisation alone. |
| Reversibility | Open-source foundation; data and configuration on your side; model weights placed in escrow. | Open-source foundation, know-how to be documented. |
| Level 3 support | Contracted, under a service-level agreement. | According to the provider's commitments. |
| Key-person dependency | A vendor, a team, a published roadmap. | To be assessed: individual or consultancy. |
| Trajectory at 12 and 36 months | Decreasing compute footprint and cost. | Risk of cost rising with drift. |
Adoption decides, and it begins on day one
A tool that teams do not take ownership of will fail regardless of model quality. RAND and Gartner put abandonment of AI projects after the proof-of-concept stage at a high level, for causes that are mostly organisational.4,5
Where adoption actually begins
Because the pipeline is already in production, your case handlers work on their own processes from the scoping phase, without waiting for a delivery. A bespoke build reverses that order: adoption cannot begin before go-live, at the point where the budget is already spent.
- 1Scoping
- 2Simulation on your real processes
- 3Field-level correction
- 4Go-live
Adoption and specialisation, from day one
- 1Specification
- 2Build
- 3Delivery
- 4Acceptance
- 5Go-live
Adoption cannot begin before here
Specialisation compounds from scoping onwards
Every field-level correction sharpens the model on your domain. Accuracy improves while the team is learning the tool, instead of starting after delivery.
The business workload exists on both sides
Rule configuration, field-level correction, per-batch acceptance, dual running during the transition: comparable in both approaches, and shrinking as model accuracy rises.
The difference lies elsewhere
With a vendor it is the only workload you carry, whereas a bespoke build adds the model maintenance team on top.
When a bespoke build is the right choice
This approach is legitimate, and there are cases where we recommend it. Three conditions have to hold at the same time:
The document scope is fixed
One transaction type, a stable format, no new partners, no expected regulatory change.
Volumes are low
Automation does not have to cover the whole flow; a significant residual manual workload remains acceptable.
A permanent MLOps team already exists
At least one dedicated, durably funded role able to detect drift, re-annotate and retrain without outside help.
If all three hold, building in-house is defensible. The third deserves the most attention: it is what determines whether maintenance cost stays under control over time.
Sources
The figures in this note come from the following references. Those drawn from our internal evaluation bench are flagged as such rather than attributed to an external source.
- 1
IEEE Computer Society — Maintenance share of total software life-cycle cost
The usual 60–80% order of magnitude, carried through the software-engineering literature since the 1980s.
- 2
ISO/IEC/IEEE 14764:2022 — Software engineering — Software life cycle processes — Maintenance
International standard that establishes software maintenance as a process in its own right, planned and budgeted.
View the source - 3
D. Sculley et al., Google — Hidden Technical Debt in Machine Learning Systems
NeurIPS 2015. Establishes that model code is a small fraction of a production machine-learning system, the bulk being data, validation and monitoring infrastructure.
View the source - 4
RAND Corporation — The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed
RR-A2680-1, 2024. Puts the failure rate of AI projects at a high level, for causes that are mostly organisational rather than technical.
View the source - 5
Gartner — Gartner Predicts 30% of Generative AI Projects Will Be Abandoned After Proof of Concept by End of 2025
Press release, July 2024.
View the source - 6
Letxbe — Internal evaluation bench
Versioned test set, one metric applied to every model compared, timestamped runs. Detailed results are in the whitepaper.
Get the full paper
The detailed version — comparison matrix, evaluation protocol, method for costing maintenance and a checklist for reading a development quote — is sent on request. Leave your work address and we will send it to you directly.
Frequently asked questions on build versus buy
Fourteen short, sourced answers on maintenance, drift, specialisation and sovereignty.
Read the FAQCompare both approaches on your own processes
Thirty minutes with a Letxbe expert to map your document flow and cost out both trajectories.
Book a conversation