Skip to main content
POST
Create Agent Evaluation Test
Changes state - verify before retry Create an agent evaluation test to save a reusable reply check, tool-call check, or simulated conversation. Saving a definition does not run it or change the agent’s prompt.

Create Agent Evaluation Test

Use an agent owned by your workspace. Choose next_reply, tool_call, or simulation. Prepare a definition that matches the agent’s prompt and tools.

Minimal request

Supply name, type, and config. folder is optional. The name is limited to 200 characters and the folder to 100 characters. Names and folders are trimmed; an empty folder is stored as no folder.
For a simulation, scenario describes the caller’s task and criteria lists the expected behavior. For next_reply, provide history ending in a caller turn and criteria for the next reply. For tool_call, save the conversation history and expected tool configuration. The config object is saved as supplied, so review its contents before running.

Agent Evaluation Test Configuration

For example, a reply-check configuration can use:
Use this object as config with type: "next_reply". Match criteria and tool names to the agent under test. Review mock_strategy, mock_fallback, and any tool mocks before running; real execution can cause external actions.

Verify the result

A 201 response returns the saved test in data. Confirm its ID and configuration with List Agent Evaluation Tests, then start a run.

Errors and recovery

400 indicates an invalid body; 404 means the agent is unavailable in this workspace. Creating again can save a duplicate test. If a response is lost, list tests and check for the saved definition before retrying. Missing, expired, or invalid API credentials return 401. See API authentication.

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

id
string
required

Agent ID from List Agents.

Body

application/json
name
string
required

Test name. Surrounding whitespace is removed.

Maximum string length: 200
type
enum<string>
required
Available options:
next_reply,
tool_call,
simulation
config
object
required

Test definition for the selected type. Use a scenario and criteria for a simulation, or history and criteria for a reply check. Saved as supplied; saving does not run the test.

folder
string | null

Optional folder. Omit, send null or an empty string to use no folder.

Maximum string length: 100

Response

Successful response.

success
enum<boolean>
required
Available options:
true
error
enum<boolean>
required
Available options:
false
statusCode
integer
required
Example:

201

message
string
required
Example:

"Success"

data
object
required
timestamp
string<date-time>
required