AI scoring and rubrics · Autessa Roleplay
Knowledge checks
Knowledge checks in Autessa Roleplay compare what a trainee said against reference text you provide, such as product facts, a refund policy or an HR process, and mark each statement Correct, Incorrect or Not supported. Anything not in the reference counts as unsupported, so confident guesses are caught.
The problem it solves
A great call with a wrong fact is still a problem
A rep who promises an integration that does not exist, or an agent who quotes the wrong refund window, can sound excellent and still create a mess. Product and policy accuracy needs to be checked against the source of truth, not judged on tone.
What you get
- Paste reference text into a knowledge check criterion: what is true, and what is explicitly not included.
- Each claim the trainee made is listed with a verdict: Correct, Incorrect or Not supported.
- Anything not covered by the reference counts as not supported.
- The reference text is shown on the rubric page under "Reference text the answers are checked against".
- Weighted like any other criterion.
- Starter examples: Product accuracy in the sales rubric, Policy accuracy (refunds, SLAs) in the support rubric, and Policy and process accuracy in the HR rubric.
The run-through
How does knowledge checks work?
- SCENE 1
Write the reference
Add a knowledge check criterion and paste in the facts, including any claims a rep must never make.
- SCENE 2
Practice
The trainee makes claims during the conversation.
- SCENE 3
Check
The scorer lists each claim with its verdict and a short note.
- SCENE 4
Correct
The trainee sees exactly which statement was wrong, and the coach can adjust if needed.
The cue sheet
Knowledge check options
| Option | What it does |
|---|---|
| Reference text | The facts statements are checked against. Anything not in here counts as unsupported. |
| Weight | How much product or policy accuracy counts toward the overall score. |
| Verdicts | Correct, Incorrect, Not supported. |
The cast
Who can do what?
| Who | Can |
|---|---|
| Coach | Write and update reference text. |
| Trainee | Read the reference text on the rubric page and see verdicts on reports. |
What the AI does
Behind the scenes
The scorer reads the trainee's turns, pulls out factual claims, and checks each one against the reference text only, so the verdict reflects your source of truth rather than what the AI happens to know.
Questions
Questions about knowledge checks
What does "Not supported" mean?
The trainee said something the reference text does not cover. It might be true, but it is not something they can back up from the source.
Can I check pricing or policy answers?
Yes. Put the policy or pricing model in the reference text and every claim about it is checked.
Does the AI use outside knowledge for the check?
No. Claims are checked against the reference text you provide.
100% customizable, code and all
Make Autessa Roleplay fully your own.
Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.
Start free and make it yours