Create Evaluation Set Item

Create a new evaluation set item for a given evaluation set.

Extract (JSON Schema) Example

curl --location --request POST 'https://api-prod.extend.app/evaluation_set_items' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_TOKEN>' \
--data '{
  "evaluationSetId": "eval_set_123",
  "fileId": "file_456",
  "expectedOutput": {
    "value": {
      "invoice_number": "36995",
      "total_amount": 15735.1,
      "line_items": [
        {
          "description": "Premium Widget",
          "quantity": 5,
          "unit_price": {
            "amount": 200.00,
            "iso_4217_currency_code": "USD"
          },
          "total": {
            "amount": 1000.00,
            "iso_4217_currency_code": "USD"
          }
        },
        {
          "description": "Basic Widget",
          "quantity": 10,
          "unit_price": {
            "amount": 25.05,
            "iso_4217_currency_code": "USD"
          },
          "total": {
            "amount": 250.50,
            "iso_4217_currency_code": "USD"
          }
        }
      ]
    },
    "metadata": {},
  }
}'

Example Response

{
  "success": true,
  "evaluationSetItem": {
    "object": "evaluation_set_item",
    "id": "eval_item_1234",
    "evaluationSetId": "eval_set_123",
    "fileId": "file_456",
    "expectedOutput": {
      // Output object - see “Processor output types” for details
    },
    "createdAt": "2024-01-01T00:00:00Z",
    "updatedAt": "2024-01-01T00:00:00Z"
  }
}

Evaluation set items are the individual files and expected outputs that are used to evaluate the performance of a given processor in Extend. This endpoint will create a new evaluation set item in Extend, which will be used during an evaluation run.

Best Practices for Outputs in Evaluation Sets

1. Configure First, Output Later

• Always create and finalize your processor configuration before creating evaluation sets
• Field IDs in outputs must match those defined in your processor configuration

2. Type Consistency

• Ensure output types exactly match your processor configuration
• For example, if a field is configured as “currency”, don’t submit a simple number value

3. Field IDs

• Use the exact field IDs from your processor configuration
• Create your own semantic IDs instead in the configs for each field/type instead of using the generated ones

4. Value

• Remember that all results are inside the value key of a result object, except the values within nested structures.

Body

evaluationSetId · string · required

The ID of the evaluation set to add the item to.

fileId · string · required

The ID of the file to add to the evaluation set.

expectedOutput · object · required

The expected output of the processor when run against the file. This should be a JSON object conforming to the output type schema of the processor.

Extract (JSON Schema) Example

curl --location --request POST 'https://api-prod.extend.app/evaluation_set_items' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_TOKEN>' \
--data '{
  "evaluationSetId": "eval_set_123",
  "fileId": "file_456",
  "expectedOutput": {
    "value": {
      "invoice_number": "36995",
      "total_amount": 15735.1,
      "line_items": [
        {
          "description": "Premium Widget",
          "quantity": 5,
          "unit_price": {
            "amount": 200.00,
            "iso_4217_currency_code": "USD"
          },
          "total": {
            "amount": 1000.00,
            "iso_4217_currency_code": "USD"
          }
        },
        {
          "description": "Basic Widget",
          "quantity": 10,
          "unit_price": {
            "amount": 25.05,
            "iso_4217_currency_code": "USD"
          },
          "total": {
            "amount": 250.50,
            "iso_4217_currency_code": "USD"
          }
        }
      ]
    },
    "metadata": {},
  }
}'

Response

success · boolean

A true or false value for whether the evaluation set item was created successfully or not.

evaluationSetItem · EvaluationSetItem

An EvaluationSetItem object representing the newly created evaluation set item. See the EvaluationSetItem object for more details.

Example Response

{
  "success": true,
  "evaluationSetItem": {
    "object": "evaluation_set_item",
    "id": "eval_item_1234",
    "evaluationSetId": "eval_set_123",
    "fileId": "file_456",
    "expectedOutput": {
      // Output object - see “Processor output types” for details
    },
    "createdAt": "2024-01-01T00:00:00Z",
    "updatedAt": "2024-01-01T00:00:00Z"
  }
}