§4 · Autessa Evaluations
Response checks
Response checks evaluate each reply before it is released, so a made-up price or a leaked card number never reaches the customer.
The area
What is response checks in Autessa Evaluations?
Response checks are LLM guardrails that evaluate each reply before the customer sees it, in web chat, embedded widgets, SMS, phone calls and the API. All enabled checks run at the same time.
Rule-based checks catch made-up facts, personal data, credentials, leaked instructions and your own patterns with no model call; AI review handles judgment calls like tone and policy. Each check flags, redacts, blocks or retries, and a broken check fails closed.
Subtopics
Everything in response checks
Response checks review every agent reply before the customer sees it: grounding, sensitive data, pattern, script and AI review checks that flag, redact, block or retry.
Fact grounding check
The fact grounding check pulls every fact out of a reply and blocks or flags any that did not come from the customer, a tool, retrieved knowledge or the instructions.
Read more →Sensitive data detectors and pattern rules
Sensitive data detectors and pattern rules find PII, credentials, leaked instructions and your own formats in a reply, and can remove just the matched text.
Read more →Script rules
Script rules are short AutessaScript checks for logic a pattern cannot express, like a competitor list kept in one place.
Read more →AI review
AI review is an LLM-as-a-judge check that runs before the reply is sent, judging tone, policy and completeness against criteria you write in plain language.
Read more →Actions and release modes
Each check has one action, the strongest action wins, and how replies are released (right away, sentence by sentence or held) follows from the checks you turn on.
Read more →Testing checks and reading results
The Test checks panel shows each check result and the exact text a customer would receive, and every checked reply in history carries its own results card.
Read more →Other areas
Agent testing →
Autessa Evaluations tests an AI agent before launch by running test sets as real conversations, driven by simulated users, and grading every one.
Scoring and judges →
Every evaluated conversation is scored on four quality dimensions and, when you describe what should happen, judged against that expected outcome.
Production monitoring →
In production, every conversation is logged in full and can be scored automatically, labeled by people and compared version by version.
Grade any agent →
Transcript Grading applies the same judges to conversations from any agent, wherever it runs, with no Autessa agent required.
Developers →
Everything in Evaluations has an API and a CLI command, so grading and test runs fit into scripts and release pipelines.
100% customizable, code and all
Make Autessa Evaluations fully your own.
Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.
Start free and make it yours