AppsEvaluations

Developers · Autessa Evaluations

Human evaluation and statistics API

The human evaluation and statistics API records reviewer verdicts on agent conversations and returns success counts over a date range, broken down by version. `autessa agents submit-evaluation` attaches a human verdict and notes, and `autessa agents statistics` returns human, automated and human-first counts. Human evaluations always take precedence over automated ones.

A core AI agent evaluation capability, done properly
Start free
Record a verdict, then read the numbers
autessa agents submit-evaluation \
  --agentId K2UYSwk3Yy4i \
  --conversationId f719af72-2e49-462e-a98c-63011a5a5671 \
  --humanEvaluation '{"status":"SUCCESSFUL","notes":"Correct summary, one tool call."}'

autessa agents statistics \
  --agentId K2UYSwk3Yy4i \
  --startDate 2026-09-01T00:00:00 \
  --endDate 2026-09-30T23:59:59

The problem

Labels and scorecards belong in your own reporting

Quality reviews often happen in spreadsheets, and the numbers never reach the dashboard leadership reads. With the statistics API, your labeling tool or weekly report reads the same counts Autessa shows, and verdicts recorded from code count exactly like verdicts recorded in the web app.

What you get

  • Autessa records a human evaluation with a status of SUCCESSFUL, SUCCESSFUL_NOT_EFFICIENT, SUCCESSFUL_NOT_ALL_STEPS or UNSUCCESSFUL, plus free-form notes.
  • The evaluation time and evaluator are set server-side from the calling user.
  • Statistics return totalConversations and three views: humanEvaluation, automatedEvaluation and humanPrecedenceEvaluation.
  • The human-first view uses the human verdict wherever one exists for a conversation.
  • versionBreakdown maps each version number to its own stats, so regressions show by version.
  • Dates use the strict form yyyy-MM-ddTHH:mm:ss with no zone, interpreted as UTC.
  • Managers (multi-agent) have the same evaluation and history APIs.

How it works

How does human evaluation and statistics api work in Autessa Evaluations?

  1. Find the conversation

    Use `autessa agents conversations history` and `conversation` to read a closed conversation with its tool log and evaluations.

  2. Record a verdict

    Call submit-evaluation with the agent, the conversation and a humanEvaluation status and notes.

  3. Pull the numbers

    Call statistics with an agent and a date range to get counts by source and by version.

Every option

Request and response fields

OptionWhat it does
agentId, conversationIdThe agent and the conversation being evaluated.
humanEvaluation.statusSUCCESSFUL, SUCCESSFUL_NOT_EFFICIENT, SUCCESSFUL_NOT_ALL_STEPS or UNSUCCESSFUL.
humanEvaluation.notesFree-form reviewer notes.
startDate, endDateISO-8601 date-time bounds, yyyy-MM-ddTHH:mm:ss with no zone, as UTC.
Evaluation statstotalEvaluated, successful, successfulNotEfficient, successfulNotAllSteps, unsuccessful.
versionBreakdownPer-version stats keyed by version number.

Permissions

Who can do what?

WhoCan
Agent editor (WRITE)Submit human evaluations.
Viewer (READ)View the agent and test-run it.
Agent Admins and company AdminsSee conversations started through the API or an app.

Connected

How it connects to the rest of Evaluations

EVALUATIONS FEATURE

Human evaluation

The same verdicts, recorded in the Evaluation Details panel of the conversation viewer.

EVALUATIONS FEATURE

Version performance

The version breakdown behind the chart in Agent History.

EVALUATIONS FEATURE

Export and sync

Land full conversations and evaluations in your own tables for deeper analysis.

FAQ

Questions about human evaluation and statistics api

Do human verdicts override automated ones?

Yes. Human evaluations take precedence in statistics, and the humanPrecedenceEvaluation view shows the merged result.

Why was my date rejected?

The date form is strict: yyyy-MM-ddTHH:mm:ss with no zone. A trailing Z, a numeric offset or a bare date are rejected, and the value is read as UTC.

Can I see results per version?

Yes. versionBreakdown maps each version number to its own stats.

Try human evaluation and statistics api in Autessa Evaluations

The agents API records human evaluations and returns human, automated and human-first statistics with a per-version breakdown.

Start free

Test sets · simulated users · four-dimension scoring · response checks · grading API

100% customizable, code and all

Make Autessa Evaluations fully your own.

Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.

Start free and make it yours