Response checks · Autessa Evaluations

Script rules

A script rule is a custom code check for agent replies: a short AutessaScript that reads the reply and the conversation and decides pass or fail. Use it for logic a pattern cannot express, like a competitor list kept in one place or a rule that depends on what the customer asked. It runs on its own, with no network access, and must finish within 2 seconds.

A core AI agent evaluation capability, done properly

Start free

Script rule · No competitorsSAMPLE

const competitors = ['Acme Support', 'HelpCo', 'DeskPro'];
const text = variables.response.toLowerCase();
const found = competitors.filter(n => text.includes(n.toLowerCase()));

response.setResponse({
  pass: found.length === 0,
  reason: found.length ? 'Mentions a competitor: ' + found.join(', ') : '',
});

The problem

Some rules are business logic, not patterns

A regular expression can find a ticket number, but it cannot say "never mention these three competitors, and say why", or "only quote a refund amount if the customer asked about a refund". Those rules belong in code you can read and change. Script rules give you exactly that, inside the same check pipeline as every other response check.

What you get

Autessa script rules read variables.response, the reply being checked.
variables.segment holds the sentence being released right now; when the whole reply is checked at once it equals variables.response.
variables.history holds the conversation so far as a JSON string of turns with role, optional name and content.
The script reports with response.setResponse({ pass, reason, matches }).
The reason shows in check results, and when the action retries it is passed to the agent as feedback.
matches lists exact text from the reply to report, or to remove when the action is Remove the matched text.
A script that throws, runs longer than 2 seconds or never calls response.setResponse counts as failed.

How it works

How does script rules work in Autessa Evaluations?

Add a script rule
Click Add check and pick Script rule, then give it a clear name.
Write the logic
Read variables.response, variables.segment or variables.history and decide whether the reply passes.
Report the result
Call response.setResponse with pass, an optional reason and optional matches.
Pick an action and test
Choose flag, remove, block or retry, then run it in the Test checks panel against a sample reply.

Every option

Script rule inputs and outputs

Option	What it does
variables.response	The reply text being checked.
variables.segment	The sentence being released now; equal to the response when the whole reply is checked.
variables.history	The conversation so far, as JSON: role (user, assistant or tool), optional name, content.
pass	true if the reply is fine; false triggers the check action.
reason	Optional short explanation, shown in results and given to the agent on a retry.
matches	Optional exact pieces of the reply to report or remove.
Limits	Runs on its own, without platform calls or network access, within 2 seconds.

From code

Do it from the CLI or API

Run `autessa <command> help` for the full list of arguments.

CLI documentation →

Example: block any mention of a competitor

const competitors = ['Acme Support', 'HelpCo', 'DeskPro'];
const text = variables.response.toLowerCase();
const found = competitors.filter(name => text.includes(name.toLowerCase()));

response.setResponse({
  pass: found.length === 0,
  reason: found.length ? 'Mentions a competitor: ' + found.join(', ') : '',
  matches: found.map(name => {
    const start = text.indexOf(name.toLowerCase());
    return variables.response.substring(start, start + name.length);
  })
});

Permissions

Who can do what?

Who	Can
Agent editor (WRITE)	Write and edit script rules on the draft.
Publisher (PUBLISH)	Publish the version that carries the rule.
Viewer (READ)	Read the agent configuration and test-run it.

Connected

How it connects to the rest of Evaluations

EVALUATIONS FEATURE

Retry with feedback

With a retry action, the reason your script returns is handed to the agent so its next draft fixes the problem.

EVALUATIONS FEATURE

Pattern rules

When a regular expression is enough, a pattern rule is simpler to read and maintain.

AUTESSA PLATFORM

AutessaScript

AutessaScript is the same scripting language used for tools and automations across Autessa.

FAQ

Questions about script rules

Can a script rule call an API or look something up?

No. Script rules run on their own, without platform calls or network access, so every check stays fast and predictable.

What happens if my script has a bug?

It counts as failed and its action applies, so a broken check never lets a reply through unchecked. On Flag only, the error is recorded and the reply goes out.

Can the script see the whole conversation?

Yes. variables.history holds the conversation so far as a JSON string of turns.

Also in Response checks:Fact grounding check Sensitive data detectors and pattern rules AI review Actions and release modes Testing checks and reading results

Try script rules in Autessa Evaluations

Script rules are short AutessaScript checks for logic a pattern cannot express, like a competitor list kept in one place.

Start free

Test sets · simulated users · four-dimension scoring · response checks · grading API

100% customizable, code and all

Make Autessa Evaluations fully your own.

Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.

Start free and make it yours