AppsEvaluations

Integrations

What Autessa Evaluations connects to

Autessa Evaluations works natively with Autessa agents, managers, AutessaDB and AutessaScript, and with any other agent through transcripts, the Grading API, the CLI and signed webhooks.

Start free
POST /clients/evaluations/grade
curl -X POST https://api.autessa.com/clients/evaluations/grade \
  -H "authorization: $AUTESSA_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "agentInstructions": "You are a billing support agent. Refund duplicate charges.",
    "expectedOutcome": "The agent refunds the duplicate charge.",
    "turns": [
      { "role": "USER", "content": "I was charged twice for order 1042." },
      { "role": "ASSISTANT", "content": "I refunded the duplicate $48. It arrives in 3-5 days." }
    ]
  }'

Connections

Built-in connections

AI agents

Every agent’s conversations, versions and tools feed evaluations; test runs use the real agent.

Multi-agent managers

Manager conversations are judged on the full history, with nested sub-agent transcripts.

AI Agent Firewall

Firewall catches appear beside evaluation trends on the quality dashboards.

AutessaDB

Conversation Sync writes closed conversations and evaluations into your tables.

AutessaScript

Script rules are short AutessaScript checks with the reply, sentence and history.

Models and privacy

Evaluations run on your company default model, including your own connected provider, with PII and PHI stripping.

Phone, SMS and web chat

Response checks run in every channel; voice is graded on what the caller heard.

Any agent, via transcripts

Agents built anywhere send user, assistant, tool call and tool result turns to the Grading API.

Webhooks

Batch jobs POST a signed summary to your https URL when they finish.

API keys and CLI

Any enabled API key calls /clients/evaluations; the CLI covers evaluations, sandbox and agents.

JSON, JSONL and CSV

Upload transcripts as .json or .jsonl; export history as JSON or CSV.

Developers

The command line

Grade transcripts, manage batch jobs and run test sets from the terminal with `autessa evaluations` and `autessa sandbox`.

CLI documentation →
autessa CLI
autessa evaluations grade        # grade one transcript now
autessa evaluations jobs-create  # up to 500 in the background
autessa evaluations jobs-get     # poll progress and pass rate
autessa evaluations jobs-items   # page through results
autessa evaluations items-get    # one result, by itemId or externalId
autessa evaluations jobs-list    # your company's jobs

autessa sandbox test-sets-create
autessa sandbox policies-create
autessa sandbox runs-start
autessa sandbox runs-get

Your data

Sync evaluations into your own tables

Conversation Sync writes each closed conversation into a table you choose, with columns filled from variables like conversation ID, version, channel and language.

Conversation Sync · Table updateSAMPLE
AgentsBilling support agent, Escalations
Tableagent_conversations
Match columnconversation_id ← @@{conversationId}
Conversation JSONtranscript (with tool calls)
More columnsversion ← @@{agentVersion} · channel ← @@{channel}
UnmatchedInsert a new row
EXPORT JSONEXPORT CSV

Grade any agent today

Send your first transcript to the Grading API, or open Transcript Grading in the browser.

Start free

Test sets · simulated users · four-dimension scoring · response checks · grading API

100% customizable, code and all

Make Autessa Evaluations fully your own.

Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.

Start free and make it yours