> ## Documentation Index
> Fetch the complete documentation index at: https://dialnexa.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Create Agent Evaluation Test

> Create an agent evaluation test to save a reusable reply check, tool-call check, or simulated conversation.

Changes state - verify before retry

Create an agent evaluation test to save a reusable reply check, tool-call check, or simulated conversation. Saving a definition does not run it or change the agent’s prompt.

## Create Agent Evaluation Test

Use an agent owned by your workspace. Choose `next_reply`, `tool_call`, or `simulation`. Prepare a definition that matches the agent’s prompt and tools.

## Minimal request

Supply `name`, `type`, and `config`. `folder` is optional. The name is limited to 200 characters and the folder to 100 characters. Names and folders are trimmed; an empty folder is stored as no folder.

```json theme={null}
{
  "name": "Answers a pricing question",
  "type": "simulation",
  "config": {
    "scenario": "Ask what the product costs.",
    "criteria": [
      "Uses the pricing in the agent prompt."
    ],
    "mock_strategy": "all",
    "mock_fallback": "error"
  }
}
```

For a simulation, `scenario` describes the caller’s task and `criteria` lists the expected behavior. For `next_reply`, provide `history` ending in a caller turn and criteria for the next reply. For `tool_call`, save the conversation history and expected tool configuration. The `config` object is saved as supplied, so review its contents before running.

## Agent Evaluation Test Configuration

| Test type | Configuration to prepare |
| - | - |
| `next_reply` | `history` with `agent` and `caller` turns ending in a caller line, plus `criteria` for the next response. Optional `reply_matchers` can check contains, not\_contains, regex, or exact text. |
| `tool_call` | `history`, `expected_tool` matching the configured tool name, `expect_call`, and optional `expected_params` entries with `name`, `match` (exact, regex, or llm), and `value`. |
| `simulation` | A caller `scenario`, `criteria`, and optional `persona_traits` or `expected_outcomes` for the full conversation. |

For example, a reply-check configuration can use:

```json theme={null}
{
  "history": [{"role": "caller", "text": "What does the product cost?"}],
  "criteria": ["Uses only the pricing supplied in the agent prompt."]
}
```

Use this object as `config` with `type: "next_reply"`. Match criteria and tool names to the agent under test. Review `mock_strategy`, `mock_fallback`, and any tool `mocks` before running; real execution can cause external actions.

## Verify the result

A `201` response returns the saved test in `data`. Confirm its ID and configuration with [List Agent Evaluation Tests](https://dialnexa.com/docs/api-reference/v1/eval-tests/list.md), then [start a run](https://dialnexa.com/docs/api-reference/v1/eval-runs/create.md).

## Errors and recovery

`400` indicates an invalid body; `404` means the agent is unavailable in this workspace. Creating again can save a duplicate test. If a response is lost, list tests and check for the saved definition before retrying.

Missing, expired, or invalid API credentials return `401`. See [API authentication](https://dialnexa.com/docs/api-reference/api-fundamentals.md#authenticate-dialnexa-api-requests).

## Related endpoints

* [List Agent Evaluation Tests](https://dialnexa.com/docs/api-reference/v1/eval-tests/list.md)
* [Start Agent Evaluation Run](https://dialnexa.com/docs/api-reference/v1/eval-runs/create.md)
* [Get Agent Evaluation Run](https://dialnexa.com/docs/api-reference/v1/eval-runs/get.md)
* [Get Agent Evaluation Conversation](https://dialnexa.com/docs/api-reference/v1/eval-runs/conversation.md)


## OpenAPI

````yaml POST /v1/agents/{id}/tests
openapi: 3.0.0
info:
  title: DialNexa API
  description: Public `/v1` REST API for the DialNexa voice AI platform.
  version: 1.0.0
servers:
  - url: https://api.dialnexa.com
    description: DialNexa production API
security:
  - bearer: []
tags:
  - name: Agents
  - name: Batch Calls
  - name: Calls
  - name: Knowledge Base
  - name: Languages
  - name: LLMs
  - name: Phone Numbers
  - name: Organization Folders
  - name: Transcribers
  - name: Webhooks
  - name: Voices
  - name: Workflows
  - name: Workflow Leads
  - name: Authentication
  - name: Agent Builder
  - name: Agent Evaluations
paths:
  /v1/agents/{id}/tests:
    post:
      tags:
        - Agent Evaluations
      summary: Create Agent Evaluation Test
      description: >-
        Saves a test definition without starting a run or changing the agent
        version. Repeating the request creates another test.
      operationId: createAgentEvalTest
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
          description: Agent ID from List Agents.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/SaveAgentEvalTest'
            examples:
              request:
                value:
                  name: Answers a pricing question
                  type: simulation
                  config:
                    scenario: Ask what the product costs.
                    criteria:
                      - Uses the pricing in the agent prompt.
                    mock_strategy: all
                    mock_fallback: error
      responses:
        '201':
          description: Successful response.
          content:
            application/json:
              schema:
                type: object
                required:
                  - success
                  - error
                  - statusCode
                  - message
                  - data
                  - timestamp
                properties:
                  success:
                    type: boolean
                    enum:
                      - true
                  error:
                    type: boolean
                    enum:
                      - false
                  statusCode:
                    type: integer
                    example: 201
                  message:
                    type: string
                    example: Success
                  data:
                    $ref: '#/components/schemas/AgentEvalTest'
                  timestamp:
                    type: string
                    format: date-time
        '400':
          description: >-
            Invalid test definition or run selection. For runs, check test
            ownership, enabled tests, target version and secondary-model
            availability.
        '401':
          description: Missing, expired or invalid API key.
        '404':
          description: Agent, test, run or conversation was not found in this workspace.
      security:
        - bearer: []
components:
  schemas:
    SaveAgentEvalTest:
      type: object
      properties:
        name:
          type: string
          maxLength: 200
          description: Test name. Surrounding whitespace is removed.
        folder:
          type: string
          maxLength: 100
          nullable: true
          description: >-
            Optional folder. Omit, send null or an empty string to use no
            folder.
        type:
          type: string
          enum:
            - next_reply
            - tool_call
            - simulation
        config:
          type: object
          additionalProperties: true
          description: >-
            Test definition for the selected type. Use a scenario and criteria
            for a simulation, or history and criteria for a reply check. Saved
            as supplied; saving does not run the test.
      required:
        - name
        - type
        - config
    AgentEvalTest:
      type: object
      properties:
        id:
          type: string
        agent_id:
          type: string
        name:
          type: string
        folder:
          type: string
          nullable: true
        type:
          type: string
          enum:
            - next_reply
            - tool_call
            - simulation
        config:
          type: object
          additionalProperties: true
        template_id:
          type: string
          nullable: true
        enabled:
          type: boolean
        last_result:
          type: object
          properties:
            run_id:
              type: string
            passed:
              type: integer
            total:
              type: integer
            finished_at:
              type: string
              format: date-time
              nullable: true
            version_number:
              type: integer
              nullable: true
            target:
              type: string
              nullable: true
          nullable: true
        created_at:
          type: string
          format: date-time
        updated_at:
          type: string
          format: date-time
      required:
        - id
        - agent_id
        - name
        - type
        - config
        - enabled
  securitySchemes:
    bearer:
      scheme: bearer
      type: http

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.