AI scoring and rubrics · Autessa Roleplay
Rubric builder
The Autessa Roleplay rubric builder is where you write the sales scorecard, or support or HR scorecard, a roleplay is graded against: criteria with weights, rating anchors, a pass mark, a rating scale and a scoring mode. Every session is scored by AI against your rubric, not a generic one.
The problem it solves
A score is only as good as the rubric behind it
Teams already have a methodology, whether it is MEDDIC, SPIN, a support standard or an HR policy. AI scoring that ignores it produces numbers nobody trusts. The rubric has to be yours, written in your words, and transparent to the people being scored.
What you get
- Name, description and pack for each rubric.
- A pass mark: the weighted percentage a trainee needs to meet the bar.
- A rating scale: rated criteria are scored from 1 to the number you choose.
- Scoring mode: one AI call per criterion (more thorough) or one call for the whole rubric (faster).
- Criteria with a name, a kind, a weight and an instruction to the scorer, reordered with up and down.
- Anchors for what a 1 looks like and what a top score looks like.
- The rubric page shows each weight as a percentage of the total and roughly how many AI tasks a session costs.
- Old reports keep the rubric they were scored against, so editing a rubric never rewrites the meaning of past reports.
The run-through
How does rubric builder work?
- SCENE 1
Describe the rubric
Name it, describe what it is for, and pick a pack.
- SCENE 2
Set the bar
Choose the pass mark and the top of the rating scale.
- SCENE 3
Add criteria
For each one, choose a kind, a weight and write what the scorer looks for, plus anchors for rated criteria.
- SCENE 4
Choose a scoring mode
One call per criterion for depth, or one call for the whole rubric for speed.
- SCENE 5
Attach it
Pick the rubric on any scenario. The rubric page lists every scenario that uses it.
The cue sheet
Rubric options
| Option | What it does |
|---|---|
| Name and description | Coaches see the description when picking a rubric for a scenario. |
| Pass mark (%) | The weighted score a trainee needs to pass. |
| Rating scale tops out at | Rated criteria are scored from 1 to this number. |
| How it is scored | One AI call per criterion, or one AI call for the whole rubric. |
| Criterion weight | Relative; the rubric page shows the percentage each one works out to. |
| What the scorer looks for | Written as an instruction to the scorer, not a description of the skill. |
| Anchors | What a 1 looks like and what a top score looks like. |
| Cost per session | Shown on the rubric page as about how many AI tasks a session takes. |
The cast
Who can do what?
| Who | Can |
|---|---|
| Coach | Create, edit and turn off rubrics. |
| Trainee | Read rubrics in the library to see how they will be scored. |
What the AI does
Behind the scenes
In per-criterion mode, the AI scorer reads the transcript once for each criterion and then writes a summary, so every criterion gets full attention. Single-call mode scores the whole rubric in one pass.
Connected
How it connects to the rest of Roleplay
Questions
Questions about rubric builder
Can I use my own sales methodology?
Yes. Write criteria for any methodology. A framework criterion holds the parts of a named method such as MEDDIC or SPIN, each rated on its own.
How is the overall score calculated?
Top-level scored criteria are weighted into a 0 to 100 percentage. Feedback-only criteria never move it, and a framework's parts are averaged into one score.
What is the difference between the two scoring modes?
One call per criterion is more thorough; one call for the whole rubric is faster and uses fewer AI tasks.
If I edit a rubric, do old scores change?
No. Each session keeps the rubric it was scored against.
100% customizable, code and all
Make Autessa Roleplay fully your own.
Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.
Start free and make it yours