Scoring and judges · Autessa Evaluations
Groundedness, completeness, faithfulness and efficiency
Autessa Evaluations scores every evaluated conversation on four dimensions: groundedness, task completeness, argument faithfulness and efficiency. Each dimension comes with a written justification, and together they set the conversation’s status: Successful, Successful (Not Efficient), Successful (Not All Steps) or Unsuccessful.
The problem
“Did it go well?” is not one question
An agent can sound confident and be wrong, finish the task by guessing a tool argument, or get there with five unnecessary tool calls. A single thumbs-up hides all of that. Four separate dimensions tell you which kind of failure happened, and the justification tells you where.
What you get
- Groundedness: are the agent’s claims backed by tools, documents or context, with nothing made up?
- Task completeness: did the agent do everything the user asked?
- Argument faithfulness: did every tool argument come from the conversation? A fabricated argument makes the evaluation unsuccessful.
- Efficiency: were tools used without extra or repeated calls?
- Each dimension has a badge (such as Grounded, Complete, All faithful, Efficient) and a written justification.
- Retrieved documents count as grounding, and interrupted voice replies are graded on what the caller actually heard.
- The same scoring is used for live conversations, test runs and Transcript Grading.
How it works
How does groundedness, completeness, faithfulness and efficiency work in Autessa Evaluations?
The judge reads everything
The user messages, the agent’s replies, every tool call with its arguments and every tool result, and the agent’s instructions.
Each dimension is scored
With a short justification that points to the part of the conversation behind it.
A status is set
Successful, Successful (Not Efficient), Successful (Not All Steps) or Unsuccessful.
You read it in context
The Evaluation Details panel sits beside the transcript, so every claim can be checked.
Every option
Evaluation statuses
| Option | What it does |
|---|---|
| Successful | Did the whole task, correctly and efficiently. |
| Successful (Not Efficient) | Did the whole task, but used its tools wastefully (extra or repeated calls). |
| Successful (Not All Steps) | Made progress, but skipped or didn’t finish part of the task. This does not pass. |
| Unsuccessful | Made claims nothing backed up, didn’t do the task, or used a tool argument it made up. |
Permissions
Who can do what?
| Who | Can |
|---|---|
| Anyone who can read the agent | See automated scores and justifications on conversations they can see. |
| Write access | Record a human evaluation that overrides the automated status. |
Where the judge comes in
How the model is used here
A model judge produces the scores and justifications. It runs on your company default model, so if you connect your own model provider, evaluations can run on it, and company-wide PII and PHI stripping applies to evaluations too.
How grading works →Connected
How it connects to the rest of Evaluations
EVALUATIONS FEATURE
Expected-outcome judge
Add success criteria and a second judge decides pass or fail.
EVALUATIONS FEATURE
Fact grounding check
Groundedness after the fact, plus a check that stops ungrounded facts before they are sent.
AUTESSA PLATFORM
AI runtime
Choose the company default model that runs evaluations.
FAQ
Questions about groundedness, completeness, faithfulness and efficiency
What does groundedness mean?
Whether every claim the agent made is backed by a tool result, a retrieved document or the conversation context. A made-up fact makes a reply ungrounded.
What is argument faithfulness?
Whether every argument the agent passed to a tool came from the conversation. Inventing an order number or an email to call a tool makes the evaluation unsuccessful.
Does a partly done task pass?
No. Successful (Not All Steps) means part of the task was skipped, and it does not pass. Successful and Successful (Not Efficient) pass.
Can I add my own criteria?
Yes, two ways. Evaluation notes give the judge a rubric for task completeness, and an expected outcome adds a second judge with your own success criteria.
100% customizable, code and all
Make Autessa Evaluations fully your own.
Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.
Start free and make it yours