Precision and recall are the two paired metrics that measure extraction quality by separating wrong values from missed values. Precision is the share of extracted values that are correct. Recall is the share of the values that should have been captured that were. In CRE document extraction, they reveal whether a model hallucinates fields or silently skips them.
How Are Precision and Recall Calculated?
Precision and recall are calculated from three counts: true positives, false positives, and false negatives. A true positive is a field the model extracted correctly. A false positive is a value it produced that is wrong or not present. A false negative is a field it should have extracted but missed. Together these counts split every extraction outcome.
The formulas are:
Precision = TP / (TP + FP), the share of extracted values that were correct.
Recall = TP / (TP + FN), the share of values that should have been extracted that were captured.
F1 score = 2 x (Precision x Recall) / (Precision + Recall), the harmonic mean of the two.
True negatives, fields correctly left blank, are excluded, because correctly ignoring an absent field does not demonstrate extraction capability (LlamaIndex). The F1 score uses a harmonic mean rather than a simple average, so it punishes imbalance: a model with precision 1.0 and recall 0.0 has an arithmetic mean of 0.5 but an F1 of 0.
Count | Meaning in extraction |
True positive (TP) | A field extracted with the correct value |
False positive (FP) | A value produced that is wrong or was not in the document |
False negative (FN) | A field present in the document that the model missed |
True negative (TN) | A field correctly left blank; excluded from these metrics |
Why Precision and Recall Matter
Precision and recall matter because a single accuracy number hides which kind of error a model makes, and the two errors have different costs. Low precision means the model invents or misreads values that a reviewer must catch. Low recall means it silently omits values that no one sees are missing. In CRE, a missed co-tenancy clause is more dangerous than a visibly wrong rent.
The two metrics trade off against each other. Pushing a model to capture more fields raises recall but tends to lower precision, because it produces more values overall and more of them are wrong; tightening it to produce only sure values raises precision but lowers recall as it skips uncertain fields (Built In). This is why F1 exists: it forces a model to balance the two rather than win on one. For an operator, the choice is deliberate. A rent-roll ingest may favor recall so no unit is dropped, while an automated covenant check may favor precision so no false alarm reaches a lender.
Example
Precision and recall are clearest computed on a real extraction run. A model processes a lease abstract where the ground truth contains 50 fields. It returns 48 values. Of those, 45 are correct, 3 are wrong, and 5 true fields were never returned at all.
Quantity | Count |
Fields in ground truth | 50 |
Values the model returned | 48 |
True positives (correct values) | 45 |
False positives (wrong values) | 3 |
False negatives (missed fields) | 5 |
The calculation walkthrough:
Precision = TP / (TP + FP) = 45 / (45 + 3) = 45 / 48 = 0.938, or 93.8%.
Recall = TP / (TP + FN) = 45 / (45 + 5) = 45 / 50 = 0.900, or 90.0%.
F1 = 2 x (0.938 x 0.900) / (0.938 + 0.900) = 2 x (0.844 / 1.838) = 0.918, or 91.8%.
The model reads correctly when it commits (93.8% precision) but drops one field in ten (90.0% recall). The F1 of 91.8% sits between them. An operator worried about missed clauses would look straight at that 90.0% recall, because the five silent omissions are the fields no reviewer is prompted to check.
Variations and Edge Cases
Precision and recall shift meaning with how you define a match and how you aggregate across fields. The variants below change what the same two numbers measure.
Variant | Behavior |
Exact-match | A value counts as correct only if every character matches; strict |
Fuzzy or partial match | Near-matches count as correct; raises both metrics, hides small errors |
Per-field precision/recall | Computed separately for each field type, such as base rent |
Micro-average | Pools all fields into one count; common fields dominate the score |
Macro-average | Averages per-field scores equally; rare fields count as much as common ones |
Precision and Recall vs Accuracy
Precision and recall are often confused with accuracy, but accuracy collapses both into one number and hides the difference between them. Accuracy is the overall share of extractions that are correct. Precision isolates how often the model is right when it commits to a value. Recall isolates how often it captures values that were present.
Put simply, accuracy is the summary and precision and recall are the diagnosis. A model can post high accuracy while quietly failing on one axis: strong precision but weak recall means it is trustworthy on what it returns yet leaves fields silently blank. Accuracy alone would not surface that gap, which is why extraction is reported on both metrics rather than one.
Frequently Asked Questions
What is the difference between precision and recall in document extraction?Precision is the share of extracted values that are correct; recall is the share of values that should have been extracted that were captured. Precision penalizes wrong values, recall penalizes missed ones. A model can be strong on one and weak on the other, which is why both are reported.
How do you calculate the F1 score for extraction?F1 score is the harmonic mean of precision and recall: F1 = 2 x (Precision x Recall) / (Precision + Recall). Because it uses a harmonic mean, it punishes imbalance, so a model cannot score well by maximizing one metric at the expense of the other.
Which matters more, precision or recall?It depends on the cost of the error. Favor precision when a wrong value is costly, such as an automated covenant check. Favor recall when a missed value is costly, such as capturing every unit in a rent roll. F1 balances the two when both matter.
Related Terms
Extraction Accuracy
Ground Truth
Field Extraction
Intelligent Document Processing
Human-in-the-Loop