A confidence score is a numerical estimate, usually between 0 and 1, that an AI extraction model attaches to each value it reads, representing how certain it is that the value is correct. In commercial real estate document extraction, every field pulled from a lease or rent roll carries its own score, and low-scoring fields are checked first.
How Does a Confidence Score Work?
A confidence score works by assigning a probability, on a 0 to 1 scale, to each extracted field based on signals like character clarity, layout position, and how well the value fits its expected type. Microsoft's Document Intelligence documentation defines confidence as the model's certainty on both the detected text and the location of the extracted value.
The score is produced per field, not per document. A base rent figure read from a clean rent-roll cell scores near 1.0. A renewal-option date buried in a narrative clause on an amended lease scores lower, because the model has less structural evidence to lean on. A downstream threshold then decides what happens: fields above it are accepted automatically, fields below it route to a human.
Signal | Effect on the score |
Character and image clarity | Clean, high-resolution text raises the score |
Layout position | Values in labeled tables score higher than values in prose |
Type fit | A number where a dollar amount is expected raises confidence |
Cross-field agreement | A value that reconciles with related fields raises confidence |
Why Confidence Scores Matter
A confidence score matters because it turns an opaque model output into a triage signal. Without it, a reviewer must re-read every extracted field to catch the wrong ones. With it, the reviewer reads only the fields the model flagged as uncertain, which is where errors concentrate. The score makes selective review possible instead of full re-keying.
The critical caveat is calibration. A raw confidence score is not a guarantee of accuracy, and uncalibrated scores tend to be overconfident. Research on LLM-based field extraction found cases where an extraction set carried an average confidence near 80% while only about 33% of those extractions were actually correct (Beyond Logprobs, arXiv 2606.24420). A confidence score is only as useful as its calibration: when 0.90 reliably means roughly nine in ten correct, the threshold becomes trustworthy; when it does not, the score misleads the reviewer it was meant to help.
Example
A confidence score is easiest to see across a single lease. An analyst extracts 60 fields from a retail lease. The model returns a confidence score for each, and the review policy sends anything below 0.85 to a human. The table below shows five representative fields from that set.
Field | Value read | Confidence | Routing |
Base rent (year 1) | $32.00 psf | 0.99 | Auto-accept |
Lease commencement | 2024-03-01 | 0.97 | Auto-accept |
Rentable square feet | 4,200 | 0.94 | Auto-accept |
Renewal option term | 5 years | 0.71 | Route to review |
Co-tenancy trigger | 60% occupancy | 0.58 | Route to review |
Across all 60 fields, 56 clear the 0.85 threshold and pass through. Four fall below it and queue for a reviewer. The analyst verifies those four, concentrated in the amended and narrative clauses, rather than re-reading all 60. The confidence score did the sorting: it spent the reviewer's attention on the 7% of fields carrying the uncertainty.
Variations and Edge Cases
A confidence score behaves differently depending on the level it is reported at and how it was calibrated. The variants below change what a given number means and how safely a threshold can rely on it.
Variant | Behavior |
Character-level | Certainty on a single recognized character; the OCR-layer signal |
Field-level | Certainty on a complete extracted value; the field is correct only if every character matches |
Calibrated score | Adjusted so 0.90 empirically means about 90% correct; safe for thresholds |
Uncalibrated score | Raw model output, often overconfident; a threshold on it is unreliable |
Type-dependent | Numeric fields tend to calibrate well; free-text fields skew overconfident |
Confidence Score vs Accuracy
A confidence score is often confused with accuracy, but they measure different things. A confidence score is the model's own prediction of whether one field is correct, assigned before any human confirms it. Accuracy is the measured rate at which extractions match a verified ground truth, known only after checking.
Put simply, confidence is a forecast and accuracy is the scoreboard. A well-calibrated model makes its confidence scores track accuracy closely, so a 0.90 field is right about 90% of the time. A poorly calibrated model can be confident and wrong, which is why confidence scores are triage signals, not proof.
Frequently Asked Questions
What is a good confidence score in document extraction?There is no universal cutoff. A useful confidence score depends on calibration and on how costly an error is: a workflow might auto-accept above 0.90 for low-stakes fields and route anything below 0.95 for high-stakes ones. The right threshold is set by the cost of a mistake, not by the number alone.
Does a high confidence score mean the extraction is correct?Not necessarily. A confidence score is the model's own estimate, and uncalibrated models can be confidently wrong. Research has found extraction sets averaging 80% confidence where only about a third of values were accurate, which is why confidence scores are used to prioritize review rather than to replace it.
How is a confidence score different from accuracy?A confidence score is a per-field prediction the model makes before verification. Accuracy is the measured share of extractions that match a verified ground truth after checking. Confidence forecasts correctness; accuracy records it.
Related Terms
Human-in-the-Loop
Optical Character Recognition
Intelligent Document Processing
Rent Roll
Offering Memorandum