Tests And Evaluations
Create Agent Evaluation Test
Create an agent evaluation test to save a reusable reply check, tool-call check, or simulated conversation.
POST
Create Agent Evaluation Test
Changes state - verify before retry
Create an agent evaluation test to save a reusable reply check, tool-call check, or simulated conversation. Saving a definition does not run it or change the agent’s prompt.
For a simulation,
Use this object as
Create Agent Evaluation Test
Use an agent owned by your workspace. Choosenext_reply, tool_call, or simulation. Prepare a definition that matches the agent’s prompt and tools.
Minimal request
Supplyname, type, and config. folder is optional. The name is limited to 200 characters and the folder to 100 characters. Names and folders are trimmed; an empty folder is stored as no folder.
scenario describes the caller’s task and criteria lists the expected behavior. For next_reply, provide history ending in a caller turn and criteria for the next reply. For tool_call, save the conversation history and expected tool configuration. The config object is saved as supplied, so review its contents before running.
Agent Evaluation Test Configuration
For example, a reply-check configuration can use:
config with type: "next_reply". Match criteria and tool names to the agent under test. Review mock_strategy, mock_fallback, and any tool mocks before running; real execution can cause external actions.
Verify the result
A201 response returns the saved test in data. Confirm its ID and configuration with List Agent Evaluation Tests, then start a run.
Errors and recovery
400 indicates an invalid body; 404 means the agent is unavailable in this workspace. Creating again can save a duplicate test. If a response is lost, list tests and check for the saved definition before retrying.
Missing, expired, or invalid API credentials return 401. See API authentication.
Related endpoints
- List Agent Evaluation Tests
- Start Agent Evaluation Run
- Get Agent Evaluation Run
- Get Agent Evaluation Conversation
Authorizations
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
Path Parameters
Agent ID from List Agents.
Body
application/json
Test name. Surrounding whitespace is removed.
Maximum string length:
200Available options:
next_reply, tool_call, simulation Test definition for the selected type. Use a scenario and criteria for a simulation, or history and criteria for a reply check. Saved as supplied; saving does not run the test.
Optional folder. Omit, send null or an empty string to use no folder.
Maximum string length:
100