Why Fine-Tuning a Model on CRE Documents Beats a Generic LLM

The default assumption in 2026 is that a frontier model, prompted well, is the strongest possible tool for reading a lease. It is not. For structured extraction from commercial real estate documents, fine-tuning CRE models on real leases, rent rolls, and offering memoranda beats a generic LLM on the three axes that decide production value: accuracy on the exact fields you need, stability of the output format, and cost per document. A generic model is a generalist that has read the whole internet and none of your leases. A fine-tuned model has read your leases and knows what a co-tenancy trigger looks like when it is buried on page 140.

The evidence is now direct. In a 2025 study, a specialized 7-billion-parameter model called Extract-0 achieved a mean reward of 0.573 on 1,000 held-out document extraction tasks, outperforming GPT-4.1 at 0.457, a 25% improvement, per the paper's own framing. It did so at a total training cost of 196 dollars, modifying only 0.53% of the model's weights. Size lost to specialization on the specific task of turning documents into structured fields.

Key Takeaways

Fine-tuning CRE models on domain documents beats a generic LLM on extraction accuracy, output-format stability, and cost per document, the three axes that determine production value.
In the 2025 Extract-0 study, a fine-tuned 7B model scored 0.573 versus GPT-4.1's 0.457 on 1,000 held-out extraction tasks, a 25% improvement at a 196 dollar training cost.
Fine-tuning lifted valid JSON output from 42.7% to 89.0% in the same study. Format stability, not just accuracy, is a core reason generic models fail in production pipelines.
In medical coding, fine-tuning raised GPT-4o mini's exact-match rate from 3.35% to 97.48% (peer-reviewed study), showing how far domain adaptation moves a generalist.
A generic model knows language. A fine-tuned model knows your document. For extraction, the second knowledge is the one that matters.

What does fine-tuning a model on CRE documents mean?

Fine-tuning a model on CRE documents means taking a base language model and continuing its training on labeled examples of real leases, rent rolls, and offering memoranda, so it learns the field names, clause structures, and formatting conventions specific to commercial real estate. The result is a model fine-tuning that outputs the exact structured fields your pipeline consumes.

A generic LLM has broad linguistic competence and no particular knowledge of your document set. Ask it to extract a base year gross-up provision and it will produce a plausible answer shaped by everything it has read, which is mostly not commercial leases. A fine-tuned model has been shown thousands of examples where the correct answer was labeled, so it has internalized what these documents contain and how their fields should be typed and named.

Modern fine-tuning does not require retraining the whole model. Techniques like Low-Rank Adaptation, or LoRA, adjust a small fraction of the weights. In the Extract-0 study, that fraction was 0.53%, 40.4 million of 7.66 billion parameters. That is why a specialized model can now be built for a few hundred dollars rather than a research budget, and why the old objection that fine-tuning is too expensive to justify no longer holds.

Why does a fine-tuned model beat a generic LLM on extraction?

A fine-tuned model beats a generic LLM on extraction because extraction is a narrow, well-defined transformation task, and narrow tasks reward specialization over scale. The generic model spends its capacity on general fluency it does not need here. The fine-tuned model spends its capacity on getting your fields right, which is the only thing the task measures.

The Extract-0 study makes the pattern concrete. The base model, before fine-tuning, scored 0.232. Supervised fine-tuning raised it to 0.507. Reinforcement learning pushed it to 0.573, a cumulative 147% improvement over the base, and past GPT-4.1's 0.457. The gains did not come from a bigger model. They came from teaching a smaller one exactly what the task was. This mirrors findings across domains: a peer-reviewed medical-coding study found that fine-tuning lifted GPT-4o mini's exact code-matching rate from 3.35% to 97.48%, a swing no amount of prompting closes.

Property	Generic LLM (prompted)	Fine-tuned CRE model
Knowledge of your document set	General only	Learned from labeled examples
Extraction accuracy on target fields	Baseline	Higher (0.573 vs 0.457 in Extract-0)
Output format stability	Drifts under load or after API updates	Trained to one schema, consistent
Cost per document at volume	Higher	Lower (see worked example)
Control and versioning	Vendor-controlled	You own the weights and the eval set

Is a fine-tuned model cheaper than a generic LLM at volume?

At production volume, a fine-tuned model is usually cheaper per document, because a smaller specialized model runs at a fraction of the inference cost of a frontier API and can hold accuracy while doing so. Nanonets, comparing fine-tuned models to frontier APIs for document AI, reports processing 10,000 documents daily costs about 50,000 dollars per year on a frontier model versus roughly 5,000 dollars on a fine-tuned one.

Run the arithmetic on a CRE portfolio. Suppose a team processes 200,000 lease and rent-roll pages a year. At a frontier-API rate, using the Nanonets ratio, that class of workload lands near the tens of thousands of dollars annually in inference alone. A fine-tuned model of the Extract-0 class, trained once for a few hundred dollars and served on cheaper hardware, moves the recurring cost down by roughly an order of magnitude on the same volume. The training cost is paid once. The inference saving recurs on every document forever.

Cost is not the only axis, and it is not even the most important one. The format-stability gain matters more in a pipeline. In the Extract-0 study, valid JSON output rose from 42.7% for the base model to 89.0% after fine-tuning. A generic model that drifts its output structure under load, as Nanonets notes frontier models do, breaks the downstream parser and manufactures exceptions. A model trained to one schema does not.

When is a generic LLM still the right choice?

A generic LLM is still the right choice when volume is low, the document set is highly varied, or you have no labeled data to train on. Fine-tuning has a fixed setup cost in data labeling and evaluation, and below a certain volume that cost never amortizes. For a handful of one-off documents a month, prompt a frontier model and move on.

The honest case for the generalist is the cold start. You cannot fine-tune on data you have not labeled, and building a golden evaluation set of correctly abstracted leases takes real work. As practitioners consistently note, annotation quality outweighs raw dataset size, so a small, clean labeled set beats a large noisy one. That labeling effort is the true cost of fine-tuning, not the compute.

The strongest framing is that these are phases, not rivals. As one way to put it: a generic model is where you start, and a fine-tuned model is where you end up once the work is worth owning. Many teams run a hybrid: a generic model handles the long tail of rare document types while a fine-tuned model handles the high-volume core of leases and rent rolls where accuracy, extraction accuracy, and cost all compound. The question is not which model is smarter. It is which one has read your documents.

Frequently Asked Questions

Does fine-tuning a small model beat a large generic LLM?

On narrow extraction tasks, yes. The 2025 Extract-0 study showed a fine-tuned 7B model scoring 0.573 versus GPT-4.1's 0.457 on 1,000 held-out tasks. Specialization beats scale when the task is well-defined, because the smaller model spends all its capacity on the exact fields the task measures.

How much does it cost to fine-tune a CRE extraction model?

The compute cost is now small. The Extract-0 study reported a total training cost of 196 dollars on a single GPU. The real cost is data labeling: building a clean set of correctly abstracted documents to train and evaluate against, which requires domain expertise, not compute budget.

Will a fine-tuned model produce more consistent output than a generic LLM?

Yes. A generic model can drift its output format under load or after an API update, breaking downstream parsers. A model fine-tuned to one schema is trained to produce that structure every time. In the Extract-0 study, valid JSON output rose from 42.7% at baseline to 89.0% after fine-tuning.

When should I use a generic LLM instead of a fine-tuned model?

Use a generic LLM when volume is low, documents are highly varied, or you lack labeled training data. The fixed cost of labeling and evaluation does not amortize at low volume. Many teams run both: a generic model for the rare long tail and a fine-tuned model for the high-volume core.

Conclusion

The instinct to reach for the biggest available model is a holdover from tasks where general intelligence was the bottleneck. Document extraction is not one of those tasks. It is a narrow, measurable transformation, and narrow measurable transformations reward a model that has seen the documents over a model that has seen everything else. The evidence is now direct and cheap to reproduce: a fine-tuned 7B model beat GPT-4.1 on held-out extraction for under 200 dollars, lifted valid output from 42.7% to 89.0%, and would run at a fraction of the inference cost at volume. For a CRE team, the strategic asset is not access to a frontier model, which everyone has. It is a model that has read your leases and an evaluation set that proves it reads them right. A generic model knows language. A fine-tuned model knows your document. For extraction, the second is the knowledge that pays.