Skip to main content
POST
Start Agent Evaluation Run
Changes state - verify before retry Start an agent evaluation run to queue test conversations against a saved draft or published version. The response acknowledges queued work; it does not mean the tests passed.

Start Agent Evaluation Run

Save tests with Create Agent Evaluation Test. Every selected test must belong to this agent, and at least one must be enabled. The selected target must have a saved version. Only one queued or running run may be active for an agent.

Minimal request

Send non-empty test_ids, repeat from 1 to 20, and target set to draft or published. draft selects the newest saved version. prompt_override applies only to draft runs.
Replace k7x2m9q4wz1abc with an actual saved test ID.

Evaluation run settings and side effects

  • mode defaults to text, which skips speech recognition and synthesis. voice runs each conversation as a browser-style test call; no phone number is dialled. Evaluation test calls are not charged to the workspace wallet.
  • llm defaults to primary, keeping the version configuration and its fallback. secondary uses the enabled fallback model alone. both queues each test against both models and doubles the conversation count for the same repeat setting. Secondary and both require an enabled fallback model.
  • Disabled tests are excluded. The returned total is the number of queued conversations after model selection and repeats.
  • Simulations can cover the primary and secondary languages in one conversation. primary_language_only in the saved test configuration limits a simulation to the primary language.
  • Tools default to mocked execution: every function tool is mocked, and a tool with no mock response returns an error instead of running. mock_strategy: none runs all tools for real, selected runs the tools not selected for mocking for real, and mock_fallback: real runs tools that have no mock response. Real execution performs the external action, so review mock_strategy, mock_fallback, and mocks before running.
  • trigger defaults to manual. pre_publish labels the run; it does not publish the agent or impose a publish gate.

Verify the result

A 201 response contains the queued run in data. Store data.id and poll Get Agent Evaluation Run until it reaches a terminal status. Check failed, errored, and conversation outcomes before treating it as a pass.

Errors and recovery

400 means a selected test is missing or belongs to another agent, every selected test is disabled, the target has no version, or secondary-model evaluation is unavailable. 404 means the agent is unavailable. 409 means an active run already exists: poll that run instead. A lost response may still have queued work. List runs before retrying; another successful request creates another run and repeats its test work and any configured real tool actions. Missing, expired, or invalid API credentials return 401. See API authentication.

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

id
string
required

Agent ID from List Agents.

Body

application/json
test_ids
string[]
required

Saved test IDs belonging to this agent. Disabled tests are excluded.

Minimum array length: 1
repeat
integer
required
Required range: 1 <= x <= 20
target
enum<string>
required

draft selects the newest saved version; published selects the published version.

Available options:
draft,
published
prompt_override
string

Optional editor prompt for draft runs. Ignored for a published target.

trigger
enum<string>
default:manual

Run label. pre_publish does not publish the agent or block publishing.

Available options:
manual,
pre_publish
mode
enum<string>
default:text

Text skips speech recognition and synthesis. Voice uses the audio test-call path.

Available options:
text,
voice
llm
enum<string>
default:primary

Primary keeps the version configuration, including fallback. Secondary uses the enabled fallback model alone. Both queues each test for each model; requires an enabled fallback model. Other settings stay the same.

Available options:
primary,
secondary,
both

Response

Successful response.

success
enum<boolean>
required
Available options:
true
error
enum<boolean>
required
Available options:
false
statusCode
integer
required
Example:

201

message
string
required
Example:

"Success"

data
object
required
timestamp
string<date-time>
required