> ## Documentation Index
> Fetch the complete documentation index at: https://docs.extend.app/llms.txt
> Use this file to discover all available pages before exploring further.

# Bounding boxes

In addition to being offered in product, we include location references (when available) in the API response payload for workflow run outputs. The format of these references depends on your processor's configuration type. **Citations** are used for the recommended **JSON Schema** configuration, while **Bounding Boxes** are used for the legacy **Fields Array** configuration.

Importantly, these references are not the same as traditional OCR and we do not make a guarantee that 100% of the time our system is able to detect and return them for all extracted values. However, we are constantly improving our models and will continue to do so.

Right now location references are only available for `Extract` output fields, and are only supported for the following file/document types:

* `PDF`
* `IMG` (jpeg, png, etc)

## Citation Schema (JSON Schema Config)

<Note>
  This section is relevant for processors using the JSON Schema config type. If
  you are using the legacy Fields Array config type, please see the [Bounding
  Box Schema](#bounding-box-schema-fields-array-config) documentation. If you
  aren't sure which config type you are using, please see the [Migrating to JSON
  Schema](/product-reference/studio/migrating_to_json_schema) documentation.
</Note>

Citations are a way to reference a specific location in a document where a field is located. Citations are returned in the `metadata` object for each field. Learn more about [Metadata here](/api-reference/guides/output_types#type-definition).

The `polygon` and `referenceText` fields are only returned if the `includeBoundingBoxCitations` option is enabled in the processor config. This can be enabled through the Studio in the Build tab under "Advanced options".

The shape of the polygon is as follows:

```typescript theme={null}
export type Citation = {
  page?: number;
  referenceText?: string | null;
  polygon?: Point[];
};

type Point = {
  x: number;
  y: number;
};
```

### How to use citations

How to use the bounding values in order to place a bounding box on a file, depends heavily on the file type. In general though, you will need to take
the bounding box values we return and convert them to whatever coordinate system your rendering library uses or what you define if you are using a native Canvas
element approach to drawing them over an image for instance.

The values we return in the `polygon` field represent the coordinates of the bounding box on the image or page.
They indicate the points of the polygon that form the bounding box. You may need to convert these values into a format suitable for your rendering library.
If your rendering library uses a coordinate system based on percentages of the total image or page size, you would need to perform an additional conversion step.

For PDFs, here is an example of how to apply a transform to the value in order to be rendered using `react-pdf-viewer` or a similar library:

```typescript theme={null}
export type HighlightArea = {
  left: number;
  top: number;
  width: number;
  height: number;
  pageIndex: number;
};

function convertPolygonToHighlightArea({
  polygon,
  pageIndex,
  pageHeight,
  pageWidth,
}: {
  polygon: Point[];
  pageIndex: number;
  pageHeight: number;
  pageWidth: number;
}): HighlightArea {
  const boundingBox = polygon.reduce(
    (acc, curr) => {
      return {
        left: Math.min(acc.left, curr.x),
        top: Math.min(acc.top, curr.y),
        right: Math.max(acc.right, curr.x),
        bottom: Math.max(acc.bottom, curr.y),
      };
    },
    { left: 0, top: 0, right: 0, bottom: 0 }
  );

  // If you are able to access the page height and width of the document, use them to calculate the highlight area
  if (pageHeight && pageWidth) {
    const highlightArea: HighlightArea = {
      height: ((boundingBox.bottom - boundingBox.top) / pageHeight) * 100 + 2,
      width: ((boundingBox.right - boundingBox.left) / pageWidth) * 100 + 2,
      top: (boundingBox.top / pageHeight) * 100 - 1,
      left: (boundingBox.left / pageWidth) * 100 - 1,
      pageIndex,
    };
    return highlightArea;
  }

  const DPI = 72; // Default DPI - calibrate this based on your product

  // Otherwise you can default to a conversion based on the zoom level of the document
  const highlightArea: HighlightArea = {
    left: boundingBox.left * 9,
    top: boundingBox.top * 9,
    width: ((boundingBox.right - boundingBox.left) * DPI) / 8,
    height: ((boundingBox.bottom - boundingBox.top) * DPI) / 8,
    pageIndex,
  };
  return highlightArea;
}
```

## Bounding Box Schema (Fields Array Config)

<Note>
  This section is relevant for processors using the Fields Array config type. If
  you are using the JSON Schema config type, please see the [Citation
  Schema](#citation-schema-json-schema-config) documentation. If you aren't sure
  which config type you are using, please see the [Migrating to JSON
  Schema](/product-reference/studio/migrating_to_json_schema) documentation.
</Note>

The default bounding box using heuristic based matches only supports the following field types:

* `date` fields
* `string` fields
* `signature` fields
* `array` fields (on nested string fields)
* `object` fields (on nested string fields)

If you have selected "Bounding box citations" in the extraction settings in the Extend Studio or the `includeBoundingBoxCitations` option is set to `true` in the processor config, you will be able to use bounding boxes for all additional field types. These bounding boxes will have significantly higher coverage.

* `enum` fields
* `number` fields
* `boolean` fields
* `null` fields - If a field is declaratively null (e.g. an empty form input) this will be returned as a bounding box reference. If there is no declarative indication of null, bounding boxes will not be returned.

The shape of the bounding box reference is as follows:

```typescript theme={null}
type BoundingBox = {
  /**
   * The left most position of the bounding box
   */
  left: number;
  /**
   * The top most position of the bounding box
   */
  top: number;
  /**
   * The right most position of the bounding box
   */
  right: number;
  /**
   * The bottom most position of the bounding box
   */
  bottom: number;
};
```

### How to use bounding boxes

How to use the bounding values in order to place a bounding box on a file, depends heavily on the file type. In general though, you will need to take
the bounding box values we return and convert them to whatever coordinate system your rendering library uses or what you define if you are using a native Canvas
element approach to drawing them over an image for instance.

The values we return in the BoundingBox type represent the coordinates of the bounding box on the image or page.
They indicate the top, bottom, left, and right extremities of the box. You may need to convert these values into a format suitable for your rendering library.
If your rendering library uses a coordinate system based on percentages of the total image or page size, you would need to perform an additional conversion step.

For PDFs, here is an example of how to apply a transform to the value in order to be rendered using `react-pdf-viewer` or a similar library:

```typescript theme={null}
export type HighlightArea = {
  left: number;
  top: number;
  width: number;
  height: number;
  pageIndex: number;
};

function convertBoundingBoxToHighlightArea({
  boundingBox,
  pageIndex,
  pageHeight,
  pageWidth,
}: {
  boundingBox: BoundingBox;
  pageIndex: number; // The index of the page in the document (0 indexed)
  pageHeight: number;
  pageWidth: number;
}): HighlightArea {
  // If you are able to access the page height and width of the document, use them to calculate the highlight area
  if (pageHeight && pageWidth) {
    const highlightArea: HighlightArea = {
      height: ((boundingBox.bottom - boundingBox.top) / pageHeight) * 100 + 2,
      left: (boundingBox.left / pageWidth) * 100 - 1,
      width: ((boundingBox.right - boundingBox.left) / pageWidth) * 100 + 2,
      top: (boundingBox.top / pageHeight) * 100 - 1,
      pageIndex,
    };
    return highlightArea;
  }

  const DPI = 72; // Default DPI - calibrate this based on your product

  // Otherwise you can default to a conversion based on the zoom level of the document
  const highlightArea: HighlightArea = {
    left: boundingBox.left * 9,
    top: boundingBox.top * 9,
    width: ((boundingBox.right - boundingBox.left) * DPI) / 8,
    height: ((boundingBox.bottom - boundingBox.top) * DPI) / 8,
    pageIndex,
  };
  return highlightArea;
}
```