Document AI economics are decided by inference latency and cost per document, not by the accuracy figure on the datasheet. Accuracy tells you whether an extraction is right. Latency and token cost tell you whether you can afford to run it across a data room of leases under a diligence clock. Two systems can post the same accuracy and differ by an order of magnitude in throughput and cost per page. The one you can run a thousand times a day is the one whose economics you understood before you bought it. Accuracy is a gate you pass once. Latency is a bill you pay on every document, forever.
Key Takeaways
Accuracy is a one-time quality gate; inference latency and token cost are recurring charges paid on every document, so they, not accuracy, govern what a pipeline can run at scale.
Stanford's HAI AI Index 2025 found the price to query a GPT-3.5-level model fell from $20.00 to $0.07 per million tokens between November 2022 and October 2024, a decline of more than 280x.
Epoch AI (Cottier et al., 2025) measured LLM inference prices falling 9x to 900x per year depending on the task, with a median near 50x per year, so per-token cost is the fastest-moving and least binding variable.
Latency, not price, is the ceiling: decode runs one output token at a time, so a long structured extraction is throughput-bound regardless of how cheap the tokens get.
In a worked example below, model inference costs under nine cents per offering memorandum while a single human exception review costs roughly one hundred times that, which is where the economics actually live.
What Actually Drives Document AI Economics?
Document AI economics are driven by three inputs: how many tokens a document consumes, how long the model takes to produce the output, and how often a human has to touch the result. Accuracy influences only the third. Token count sets cost, latency sets throughput, and the human review rate sets the largest line on the bill.
The datasheet leads with accuracy because it is a single number that sounds like a guarantee. The invoice does not care. When you push a portfolio of documents through an extraction pipeline, the cost you pay is a function of tokens in, tokens out, and wall-clock time per document, multiplied by volume. Intelligent document processing at portfolio scale is a throughput problem wearing an accuracy costume. The drivers below are the ones that show up on the invoice.
Driver | What it controls | Scales with |
|---|---|---|
Input tokens (context length) | Prefill latency and input cost | Document length and page density |
Output tokens | Decode latency (usually dominant) and output cost | Size of the extraction schema |
Concurrency and batching | Documents processed per hour | Infrastructure and rate limits |
Human review time | Cost per exception | The straight-through rate |
Notice that accuracy is not a row. It sets how often the last row fires, but it is not itself a cost driver. A large language model that is two points more accurate but twice as slow is more expensive to run, not less. The economics reward the system that clears volume, and volume is governed by latency and the exception rate.
Why Does Latency Cap Throughput More Than Accuracy Does?
Latency caps throughput because generation is sequential: the model produces one output token at a time, and each token waits on the one before it. Input is processed in a single parallel prefill pass, but the structured extraction a document AI system returns is output, and output cannot be parallelized within a single request. Cheaper tokens do not make this faster.
Inference-serving research separates two clocks. Time to first token measures the prefill phase, when the model reads the whole input at once; representative ranges from inference-benchmarking studies put this in the tens to low hundreds of milliseconds for typical prompts. The second clock, tokens per second during decode, governs everything after, and representative sustained rates land in the range of 20 to 200 tokens per second per stream depending on model and hardware. A survey of LLM inference-serving systems (arXiv, 2025) frames the tradeoff plainly: optimizing for throughput sacrifices per-request latency and the reverse, which is why a single latency number never describes a pipeline.
The consequence for document AI is direct. A long structured output, the kind an underwriter needs, is decode-heavy, so it is bound by tokens per second, not by token price. You can drive the per-token cost toward zero, as the market has, and the wall-clock time to emit three thousand tokens of structured fields barely moves. That is why the cheap variable and the binding variable are not the same variable. This is the split most buyers miss: they negotiate the price that is collapsing on its own and ignore the latency that sets their real ceiling.
What Does One Document Actually Cost?
One document costs cents in model inference and dollars in everything around it. The way to see this is to derive it from stated inputs rather than trust a headline. The example below uses a sixty-page offering memorandum and assumptions labeled as such; swap your own numbers and the shape of the answer holds.
Assume a sixty-page OM at roughly 500 tokens per page, giving 30,000 input tokens, and a structured extraction of 3,000 output tokens. Assume a representative mid-2026 frontier-tier price of $2.00 per million input tokens and $8.00 per million output tokens, a level consistent with the steep declines Epoch AI documents. The model cost is then:
Input: 30,000 / 1,000,000 x $2.00 = $0.060
Output: 3,000 / 1,000,000 x $8.00 = $0.024
Model cost per document: about $0.084, under nine cents.
Now the throughput side, derived from latency rather than price. At a sustained 50 output tokens per second, emitting 3,000 tokens takes 60 seconds of decode, plus a few seconds of prefill, call it 65 seconds per document on a single stream. One stream clears roughly 55 documents per hour. To process a 500-document portfolio in an hour, you need about nine concurrent streams, not a lower token price. Cost fell to nine cents on its own; throughput is an infrastructure decision.
Line | Value | Source of the number |
|---|---|---|
Model cost per document | ~$0.084 | Derived from token counts and assumed price |
Single-stream throughput | ~55 docs/hour | Derived from 3,000 tokens at 50 tokens/sec |
Human exception review | ~$10.00 | 8 min at a $75/hour fully loaded rate |
Cost ratio, review to inference | ~119x | $10.00 / $0.084 |
The punchline is in the last two rows. A single human review, at eight minutes of an analyst's time at a $75 per hour fully loaded rate, costs about $10.00, roughly one hundred and twenty times the model inference cost. The accuracy figure you shopped for changes how often that $10.00 line fires. It does not change the nine cents. So the lever with real economic weight is not the accuracy that trims exceptions from 8% to 6%; it is the latency and concurrency that decide whether the portfolio clears in an hour or a day, and the straight-through rate that decides how many $10.00 reviews you pay for at all.
Where Does Human Review Change the Economics?
Human review changes the economics more than any other variable because it is the only line priced in dollars while the rest are priced in cents. Every document that clears without a human touch costs cents. Every document that routes to an exception queue costs a person's time, which is one to two orders of magnitude more. The straight-through rate is the real cost driver.
This reframes what accuracy buys. A two-point accuracy gain matters only insofar as it lifts the straight-through rate and removes human reviews from the queue. If a more accurate model is slower or routes the same share of documents to people, it saved you nothing that shows up on the invoice. The messy-PDF reality of the documents CRE runs on means some exception rate is permanent, so the economics reward the system that makes each unavoidable review fast and traceable, not the one with the highest benchmark. A cited, well-structured output a reviewer confirms in ninety seconds beats an uncited one that takes eight minutes, even at identical accuracy, because it moves the expensive line, not the cheap one.
Frequently Asked Questions
Why do document AI economics depend on latency instead of accuracy? Because accuracy is a one-time quality gate while latency is a recurring charge on every document. Decode runs one token at a time, so latency caps how many documents you clear per hour. Accuracy only sets how often a human review fires, and that review is where the real cost sits.
Is the falling price of tokens making document AI cheaper to run? Token price is falling fastest, 9x to 900x per year by Epoch AI's measurement, which is exactly why it is not the binding constraint. When the cheap variable keeps getting cheaper on its own, your real cost ceiling moves to latency-bound throughput and human exception review, neither of which falls with token price.
How do I estimate cost per document for an extraction pipeline? Multiply input tokens by the input price and output tokens by the output price, then add the expected human review cost weighted by your exception rate. In the worked example above, model inference is under nine cents per document while one human review is about ten dollars, so the exception rate dominates the estimate.
Conclusion
The industry sells accuracy because it is a clean number that fits on a slide. Document AI economics are set elsewhere. Token cost is collapsing on its own and will keep collapsing; it is the least of your worries. Latency sets how many documents you clear per hour, and the straight-through rate sets how many human reviews you pay for. Those two numbers, not accuracy, decide whether a pipeline pays for itself at scale.
For the operator, the takeaway is to change what you interrogate at procurement. Ask for throughput at your document lengths and your concurrency, not a benchmark accuracy in a vacuum. Ask what share of documents clears without a human and how fast a reviewer confirms the rest. A system that is a point less accurate but twice as fast and half as reliant on people is cheaper to run, every day, on every document. Accuracy gets you in the door. Latency and human review decide what you can afford once you are inside.