The F1 score is the harmonic mean of a model's precision and recall, expressed on a scale from 0 to 1. It condenses two competing error rates into one number, rewarding a model only when it is both precise and complete. In document extraction it grades field-level correctness against a ground truth.
How the F1 Score Works
The F1 score works by taking the harmonic mean of precision and recall, computed as F1 = 2PR / (P + R). Precision P is the share of predicted positives that are correct. Recall R is the share of actual positives that are found. The harmonic mean, not the arithmetic mean, is the point of the metric.
The harmonic mean punishes imbalance because it pulls toward the smaller of the two inputs. A high F1 is possible only when precision and recall are both high (Wikipedia, F-score). Consider a lease-clause detector at precision 0.95 and recall 0.50. The arithmetic mean is 0.725, which flatters the model. The F1 is 2 x 0.95 x 0.50 / (0.95 + 0.50) = 0.95 / 1.45 = 0.655. The metric refuses to average away the recall gap.
The measure traces to C.J. van Rijsbergen's 1979 book Information Retrieval, where he defined an effectiveness function and motivated the harmonic mean through the principle of decreasing marginal relevance. The name F-measure entered wide use through the Message Understanding Conferences of the early 1990s (van Rijsbergen, 1979; Manning, Raghavan, Schütze, Introduction to Information Retrieval, 2008).
Why the F1 Score Matters
The F1 score matters because a single misleading number decides whether an extraction model is trusted or reviewed. Precision and recall move in opposite directions as a threshold shifts. F1 forces a model to earn both, which is why information retrieval and extraction tasks are evaluated on it rather than on raw accuracy, especially when positives are rare (Wikipedia, F-score).
In commercial real estate document work, this maps directly to underwriting risk. A rent-roll extractor with high precision but low recall looks clean while silently dropping units, understating income. High recall with low precision floods a reviewer with false hits. The F1 is the one figure that flags either failure. A model reporting 0.98 accuracy but 0.60 F1 is not ready to remove a human from the loop.
Example
The F1 score is clearest across a range of precision and recall pairs on the same task. Each row below applies F1 = 2PR / (P + R). Note how balanced inputs hold their value while a single weak input collapses the score, even when the other is near perfect.
Precision | Recall | Calculation | F1 Score |
|---|---|---|---|
0.90 | 0.90 | 2(0.81) / 1.80 | 0.900 |
0.70 | 0.70 | 2(0.49) / 1.40 | 0.700 |
1.00 | 0.50 | 2(0.50) / 1.50 | 0.667 |
0.95 | 0.50 | 0.95 / 1.45 | 0.655 |
0.99 | 0.01 | 0.0198 / 1.00 | 0.020 |
The last row is the lesson. Precision of 0.99 with recall of 0.01, a model that fires almost never but is right when it does, scores an F1 of 0.020. An arithmetic mean would report 0.50 and hide the failure. The harmonic mean does not.
Variations and Edge Cases
The F1 score is the balanced case of a broader family, and it changes meaning when classes multiply. The variants below shift what the single number certifies.
Variant | Behavior |
|---|---|
F-beta score | Fβ = (1 + β²)PR / (β²P + R). β above 1 weights recall, β below 1 weights precision |
F0.5 | Halves recall's weight; used when false positives cost more |
F2 | Doubles recall's weight; used when missed positives cost more |
Macro F1 | Unweighted mean of per-class F1; every field type counts equally regardless of frequency |
Micro F1 | Pools true positives, false positives, and false negatives across all classes, then applies the formula once; dominated by frequent classes |
Macro F1 and micro F1 diverge sharply on imbalanced field sets. Macro F1 exposes weak performance on rare fields like co-tenancy clauses. Micro F1 rewards volume, so a model strong on common fields can post a high micro score while failing the rare ones (Manning, Raghavan, Schütze, 2008). The edge case at both ends: if precision or recall is 0, F1 is 0, and if a class has no true instances, per-class F1 is undefined.
F1 Score vs Accuracy
The F1 score is often confused with accuracy, but they answer different questions. Accuracy is (TP + TN) / total, the share of all predictions correct. F1 is the harmonic mean of precision and recall and ignores true negatives entirely. On imbalanced data the gap is decisive.
Suppose 5 of 100 documents contain a default clause and a model predicts none. Accuracy is 95 / 100 = 0.95, since it is right on every clean document. Recall is 0, so F1 is 0. Accuracy calls the model excellent; F1 calls it useless. This is why extraction and retrieval, where the positive class is rare, are judged on F1, and accuracy is reported only as context (Google for Developers, Classification metrics).
Frequently Asked Questions
What is a good F1 score? It depends on the task and the base rate, but on a scale of 0 to 1, values above 0.90 are strong for production extraction and above 0.95 are excellent. There is no universal cutoff, because a score is only meaningful against a stated benchmark dataset.
Why use the harmonic mean instead of the arithmetic mean? The harmonic mean pulls toward the smaller value, so a high F1 requires both precision and recall to be high. The arithmetic mean lets one strong number mask a weak one, which is exactly the failure the F1 score is designed to expose.
What is the difference between macro and micro F1? Macro F1 averages the F1 of each class with equal weight, so rare fields count as much as common ones. Micro F1 pools the counts across all classes first, so frequent fields dominate the result.
Related Terms
Precision and Recall