Agent testing · Autessa Evaluations
Eval history
Eval history is the record of every test run for an agent, on the agent’s own page. Each row shows when it ran, the test set, the version, the pass rate, the number of cases and the status, and every column is sortable.
The problem
Was the last version actually better?
Without a history, each test run is a one-off and nobody can say whether quality is going up. Eval history keeps every run beside the version it tested, so the answer is one sort away.
What you get
- Every test run for the agent, newest first by default.
- Columns for when, kind, test set, version, pass rate, cases and status.
- Every column sorts, so the best or worst run is one click away.
- Click a run to open its full results right under the table, and click again to close it.
- Rows stay current while a run is still finishing.
- Pass rates also feed the agent’s eval pass rate chart over time.
How it works
How does eval history work in Autessa Evaluations?
Open the agent
Eval history sits on the agent’s overview.
Sort by what matters
Sort by version to compare releases, or by pass rate to find the weakest run.
Open a run in place
Results expand under the table, with the same breakdowns as the run page.
Every option
Eval history columns
| Option | What it does |
|---|---|
| When | When the run started. |
| Kind | The kind of run. |
| Test set | The test set name at run start. |
| Version | Draft or the published version number tested. |
| Pass rate | The run’s pass rate once finished. |
| Cases | Conversations in the run (cases × personas). |
| Status | Pending, Running, Completed or Failed. |
From code
Do it from the CLI or API
Run `autessa <command> help` for the full list of arguments.
Permissions
Who can do what?
| Who | Can |
|---|---|
| Read | View the agent and its versions, test-run it, and read its history and results. |
| Write | Everything Read can, plus edit the draft and record human evaluations. |
| Publish | Publish and roll back versions, which is what customers get. |
| Admin | Share and delete the agent. The owner is always an Admin. |
Connected
How it connects to the rest of Evaluations
FAQ
Questions about eval history
Where is eval history?
On each agent’s overview page. It lists every test run of that agent.
Can I compare two versions?
Yes. Sort by version and compare pass rates, or open each run in place to compare breakdowns.
Can I delete a run?
Yes, from the CLI with `autessa sandbox runs-delete`, which removes the run and all its case results.
100% customizable, code and all
Make Autessa Evaluations fully your own.
Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.
Start free and make it yours