A bounding box is a rectangle defined by coordinates that marks the exact location of a piece of text, a table cell, or a figure on a document page. In commercial real estate document extraction, each extracted value carries a bounding box, so a base rent figure links back to the precise spot on the lease page where it was read.
How Does a Bounding Box Work?
A bounding box works by storing the coordinates of a rectangle that encloses a detected element on the page. Two conventions dominate: the x and y coordinates of the top-left and bottom-right corners, or the center point plus width and height, per LlamaIndex. Coordinates are either absolute pixel values or normalized between 0 and 1 for resolution independence.
Bounding boxes are produced at multiple granularities, from a whole paragraph down to a single word or table cell. Pulse AI made word-level and cell-level bounding boxes generally available in 2025, giving each token its own rectangle. Layout models like Microsoft's LayoutLM, released in 2020, combine the text with these box coordinates to understand reading order and structure, not just the characters.
Coordinate format | Definition |
Corner (x1, y1, x2, y2) | Top-left and bottom-right corners of the rectangle |
Center (xc, yc, w, h) | Center point plus width and height |
Absolute | Pixel values tied to image resolution |
Normalized | Values scaled 0 to 1, resolution independent |
Why a Bounding Box Matters
A bounding box matters because it turns an extracted value into a verifiable claim about the source document. Without coordinates, base rent is a number with no provenance; with a bounding box, a reviewer clicks the field and lands on the exact rectangle on the lease page. Mistral OCR 4, released June 23, 2026, returns bounding boxes alongside text specifically to support source-grounded citations, per sakutto.
Boxes also drive structure. Reading order, table reconstruction, and multi-column layout all depend on where each element sits, per Microsoft Document Intelligence layout documentation. A rent roll only becomes a table because the system knows which boxes align into rows and columns. Get the boxes wrong and a value lands in the wrong column, corrupting the record even when every character was read correctly.
Example
Bounding box accuracy is measured with intersection over union (IoU), the overlap area divided by the combined area of the predicted box and the ground-truth box. IoU ranges from 0 (no overlap) to 1 (perfect), per V7 and PyImageSearch.
Input | Value |
Predicted box area | 900 px squared |
Ground-truth box area | 1,000 px squared |
Overlap (intersection) area | 800 px squared |
Union area (900 + 1,000 - 800) | 1,100 px squared |
IoU (800 / 1,100) | 0.73 |
COCO pass threshold | 0.50 |
A predicted box overlaps the true text region by 800 square pixels. Union is 900 plus 1,000 minus the 800 shared, or 1,100. IoU is 800 divided by 1,100, which equals 0.73. Because 0.73 clears the COCO benchmark minimum of 0.50, this detection counts as correct, per COCO evaluation standards. The pixel areas are illustrative inputs; the IoU and pass or fail follow from them. A box that read the right characters but landed at IoU 0.30 would fail, because it cannot be trusted to tie the value to the right cell.
Variations and Edge Cases
Bounding boxes vary by what they enclose and how the text is oriented. A clean horizontal line of print is the simple case; rotated stamps and merged table cells are where boxes get hard.
Variant | Treatment |
Word-level box | One rectangle per word; finest granularity for citations |
Line or block box | Encloses a whole line or paragraph |
Cell box | Encloses one table cell for row and column alignment |
Rotated or skewed text | Axis-aligned box overshoots; a polygon fits tighter |
Overlapping boxes | Stamps or handwriting over print produce colliding rectangles |
Bounding Box vs Optical Character Recognition
A bounding box is often conflated with optical character recognition, but they answer different questions. OCR answers what the characters are; the bounding box answers where they sit. OCR reads pixels into text, and the bounding box records the coordinate rectangle that text occupies on the page.
The two travel together in every extraction pipeline. OCR without boxes gives you a wall of text with no way to trace a value back to its source or rebuild a table. Boxes without OCR give you empty rectangles. Together they yield a value, its meaning, and its exact location, which is what makes an extraction auditable.
Frequently Asked Questions
What is a bounding box in document AI?A bounding box is a coordinate rectangle that marks where a piece of text, a table cell, or a figure sits on a document page. In extraction, each value carries its bounding box so a reviewer can trace it back to the exact spot it was read from.
How is bounding box accuracy measured?Bounding box accuracy is measured with intersection over union (IoU), the overlap area divided by the combined area of the predicted and ground-truth boxes. IoU runs from 0 to 1, and a common pass threshold is 0.50, the COCO benchmark minimum, per V7 and PyImageSearch.
Why do extracted values include bounding boxes?Extracted values include bounding boxes so each value can be traced to its source location on the page, supporting audit and source-grounded citation. Mistral OCR 4 returns bounding boxes for this reason, per sakutto, and they also drive table and reading-order reconstruction.
Related Terms
Optical Character Recognition
Document Extraction
Table Extraction
Source Citation
Confidence Score