§5 · Autessa Evaluations

Grade any agent

Transcript Grading applies the same judges to conversations from any agent, wherever it runs, with no Autessa agent required.

Start free

Transcript Grading · Grade a transcriptSAMPLE

Agent instructions

You are a billing support agent. Look up the customer’s order, refund duplicate charges, and tell them when the refund will arrive.

Conversation

User: Hi, I was charged twice for order 1042.
Agent: Sorry about that! Let me check order 1042 for you.
User: Thanks. It was $48 both times.
Agent: I’ve refunded the duplicate $48 charge. It arrives in 3–5 business days.

PassedDecided by the expected outcome: only Met passes.

The area

What is grade any agent in Autessa Evaluations?

Transcript Grading applies Autessa’s judges to conversations from any agent, built on Autessa or anywhere else. Send the turns (user and assistant messages, tool calls and results) and the instructions the agent was given, plus an optional expected outcome.

Grade one in the browser in a few seconds, or send up to 500 per batch job and get a signed webhook when it finishes. Your own ids make resubmission safe.

In this area

Subtopics

Everything in grade any agent

Grade conversation transcripts from any AI agent, built on Autessa or anywhere else: one at a time in the browser or in batch jobs with signed webhooks.

Transcript Grading

Transcript Grading scores conversations from any agent, built on Autessa or anywhere else, with the same judges that score Autessa agents.

Batch grading jobs

Batch grading jobs grade up to 500 transcripts in the background, with a signed completion webhook and your own ids for safe resubmission.

Other areas

Agent testing →

Autessa Evaluations tests an AI agent before launch by running test sets as real conversations, driven by simulated users, and grading every one.

Scoring and judges →

Every evaluated conversation is scored on four quality dimensions and, when you describe what should happen, judged against that expected outcome.

Production monitoring →

In production, every conversation is logged in full and can be scored automatically, labeled by people and compared version by version.

Response checks →

Response checks evaluate each reply before it is released, so a made-up price or a leaked card number never reaches the customer.

Developers →

Everything in Evaluations has an API and a CLI command, so grading and test runs fit into scripts and release pipelines.

Start with grade any agent

Grade conversation transcripts from any AI agent, built on Autessa or anywhere else: one at a time in the browser or in batch jobs with signed webhooks.

Start free

Test sets · simulated users · four-dimension scoring · response checks · grading API

100% customizable, code and all

Make Autessa Evaluations fully your own.

Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.

Start free and make it yours