AppsEvaluations

Response checks · Autessa Evaluations

Actions and release modes

Every response check has one action that decides what happens when it fails: flag only, remove the matched text, block the reply, retry then block, or retry then send. When several checks fail on one reply, the strongest action wins. How replies are released, right away, sentence by sentence or held, follows from the checks you turn on, with no separate setting to get wrong.

Goes beyond a typical eval tool
Start free
Actions · strongest winsSAMPLE
1
Retry, then block
The agent is told what was wrong and writes a new reply; blocked if it still fails.
2
Retry, then send
Same, but the last retry is sent anyway.
3
Block the reply
The customer gets the fallback message.
4
Remove the matched text
Replaced with [REDACTED]; the rest is sent.
5
Flag only
Sent unchanged; recorded in history.

How replies are released

RIGHT AWAY · FLAG ONLYSENTENCE BY SENTENCEHOLD · AI REVIEW OR RETRY

The problem

Safety and speed usually pull in opposite directions

Holding every reply until a judge approves it keeps customers safe but makes streaming chat and voice feel slow. Sending right away keeps things fast but means you can only log mistakes after the fact. Autessa picks the release mode from your checks: as live as possible, as careful as your checks require.

What you get

  • Flag only sends the reply unchanged and records the problem in conversation history.
  • Remove the matched text replaces it with [REDACTED] and sends the rest; it is available on every type except AI review.
  • Block the reply sends the fallback message instead.
  • Retry then block tells the agent what was wrong and lets it rewrite; after the maximum retries it sends the fallback message. Retry then send sends the last reply anyway.
  • When several checks fail, a retry comes first while retries remain, then a block, then removal. Flags are always recorded.
  • Checks fail closed: a check that cannot run counts as failed and its action applies. On Flag only, the error is recorded and the reply goes out.
  • On phone and voice calls, each sentence is spoken only after it passes, and a blocked sentence is never spoken.

How it works

How does actions and release modes work in Autessa Evaluations?

  1. Pick an action per check

    Choose the action that fits the risk: flag while you learn, remove for data, block or retry for what must never go out.

  2. Set the fallback message

    Write what the customer sees when a reply is blocked. Left empty, it is "Sorry, I can't share that. Is there anything else I can help you with?"

  3. Set max retries

    Choose how many new replies the agent may write before a retry action gives up. The default is 2.

  4. Read the release note

    The note under the check list says how replies are released now, so you always know what the customer will notice.

Every option

How replies are released

OptionWhat it does
No checks, or every check is Flag onlyReplies are sent right away and checks only record problems. The customer notices nothing.
Checks that remove or block, with no AI review and no retryReplies are checked sentence by sentence before they are sent. Each sentence waits only a few milliseconds, so streaming and voice still feel live.
Any AI review that does more than flag, or any Retry actionEach reply is held until every check passes, then appears all at once.
Fallback messageWhat the customer sees when a reply is blocked.
Max retriesHow many rewrites a retry action allows; 2 by default.

From code

Do it from the CLI or API

Run `autessa <command> help` for the full list of arguments.

CLI documentation →
Agent-wide settings with three checks (Agents API)
{
  "responseChecks": {
    "fallbackMessage": "Sorry, I can't share that. Can I help with anything else?",
    "maxRegenerations": 2,
    "checks": [
      { "checkType": "GROUNDING", "name": "No made-up prices or order numbers", "action": "BLOCK" },
      { "checkType": "BUILT_IN", "name": "Sensitive data", "action": "REDACT" },
      { "checkType": "AI_REVIEW", "name": "Policy review", "action": "REGENERATE_THEN_BLOCK" }
    ]
  }
}

Permissions

Who can do what?

WhoCan
Agent editor (WRITE)Choose actions, fallback message and max retries on the draft.
Publisher (PUBLISH)Publish the version so the new actions apply to customers.
CustomerReceives only the released reply or the fallback message, never check details.

Connected

How it connects to the rest of Evaluations

EVALUATIONS FEATURE

Testing checks

See the decision, such as Redacted or Blocked, and the exact text a customer would receive before you publish.

EVALUATIONS FEATURE

AI review

The check that most often calls for a retry, because the reviewer explains what to fix.

AUTESSA PRODUCT

Autessa Receptionist

On calls, every sentence is spoken only after it passes its checks.

FAQ

Questions about actions and release modes

Why can a retry only happen in hold mode?

Because you cannot take back a sentence the customer already read or heard. That is why AI review and retry actions switch the agent to holding each reply.

What if a check breaks?

It counts as failed and its action applies, so a broken check never lets a reply through unchecked. The one exception is Flag only, which records the error and lets the reply go out.

Do more checks make replies slower?

Not one after another. Every enabled check looks at the text at the same time, so the slowest check sets the wait, not the total.

What does a caller hear when a sentence is blocked?

The fallback message. A blocked sentence is never spoken, and the agent stops that reply.

Try actions and release modes in Autessa Evaluations

Each check has one action, the strongest action wins, and how replies are released (right away, sentence by sentence or held) follows from the checks you turn on.

Start free

Test sets · simulated users · four-dimension scoring · response checks · grading API

100% customizable, code and all

Make Autessa Evaluations fully your own.

Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.

Start free and make it yours