Response checks · Autessa Evaluations
Sensitive data detectors and pattern rules
Sensitive data detectors and pattern rules are LLM guardrails for personal data, credentials and your own formats. Turn on ready-made detectors for Social Security numbers, card and bank numbers, API keys, private values the customer entered and leaked instructions, or write a regular expression for anything else. Each can remove just the matched text and send the rest of the reply.
The problem
Agents repeat what tools hand them
A tool returns a customer record and the agent reads the Social Security number back. Someone talks the agent into reciting its own instructions. A reply includes a link to an internal admin page. None of these are model errors in the usual sense; the agent is being helpful with data it should never have said out loud. Detectors catch that data in the reply itself, before it leaves.
What you get
- Autessa includes detectors for US Social Security numbers, payment card numbers that pass the card checksum, US bank account and routing numbers, and US driver license numbers.
- Detectors also find email addresses, phone numbers, IP addresses, IBANs and dates written as a date of birth.
- The API keys, tokens and private keys detector covers credentials from common cloud, developer, payment and AI providers, signed tokens and private key blocks.
- The private values detector catches any value the customer typed for a private argument in this conversation, or a value from a secret, repeated word for word.
- The instructions detector flags a long run of words copied from the agent instructions: 12 words in a row by default, and you can change it.
- The internal links detector finds links to local machines, private network addresses and internal-only domains, plus any domains you list.
- Pattern rules match your own regular expression, with standard flags, required context keywords within 80 characters, and an optional card-number checksum.
How it works
How does sensitive data detectors and pattern rules work in Autessa Evaluations?
Add a detector check
Click Add check and pick Sensitive data detectors, then turn on the detectors this agent needs.
Or write a pattern rule
Pick Pattern rule and enter a regular expression, such as an internal ticket format like TCK-\d{6}.
Cut false matches
Add required context keywords so nine digits only count near "account" or "member ID", or require the card checksum to screen out random digit runs.
Choose Remove the matched text
The matched text becomes [REDACTED] and the rest of the reply goes out, or pick block or retry when any match should stop the reply.
Every option
Detectors and pattern rule options
| Option | What it does |
|---|---|
| Social Security numbers | US Social Security numbers. |
| Card numbers | Payment card numbers that pass the card checksum. |
| Bank account and routing numbers | US bank account and routing numbers. |
| Driver's license numbers | US driver license numbers. |
| Email addresses, phone numbers, IP addresses | Any email address, phone number or IP address. |
| IBANs | International bank account numbers. |
| Dates of birth | Dates written as a date of birth. |
| API keys, tokens and private keys | Provider credentials, signed tokens and private key blocks. |
| Private values the customer entered | A private argument value or secret repeated word for word. |
| The agent's own instructions | A run of words copied from the instructions; 12 in a row by default, adjustable. |
| Internal links and addresses | Local machines, private network addresses, internal-only domains and domains you list. |
| Pattern: expression and flags | Your regular expression, with case-insensitive, multi-line and the other standard flags. |
| Pattern: required context keywords | A match counts only if one of these words appears within 80 characters of it. |
| Pattern: card-number check | A match counts only if it passes the checksum real card numbers pass. |
From code
Do it from the CLI or API
Run `autessa <command> help` for the full list of arguments.
Permissions
Who can do what?
| Who | Can |
|---|---|
| Agent editor (WRITE) | Choose detectors, write pattern rules and set actions on the draft. |
| Publisher (PUBLISH) | Publish the version so the detectors apply to live traffic. |
| Reviewer with history access | See what was caught in the Response checks card on each reply. |
Connected
How it connects to the rest of Evaluations
EVALUATIONS FEATURE
Remove only the match
Removal swaps the matched text for [REDACTED] and keeps the reply streaming sentence by sentence.
AUTESSA PRODUCT
Autessa Redact
Redact documents, recordings and transcripts with the same care your agents get on every reply.
AUTESSA PRODUCT
AI Agent Firewall
The firewall guards what comes in; response checks guard what goes out.
FAQ
Questions about sensitive data detectors and pattern rules
Can a tool echo a private value like a card number back to the customer?
Not through the AI. A private value goes straight to the tool, and if the tool result repeats it, the value is masked before the AI reads the result. The private values detector is a backstop for the rare case where the customer also typed it into the normal chat.
How do I stop the agent from revealing its instructions?
Turn on The agent's own instructions detector. It flags a long run of words copied from the instructions, 12 in a row by default.
How do I avoid matching every nine-digit number?
Use a pattern rule with required context keywords. A match then counts only when a word like "account" appears within 80 characters.
Do detectors call a model?
No. Detectors and pattern rules are rules that run instantly, in parallel with your other checks.
100% customizable, code and all
Make Autessa Evaluations fully your own.
Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.
Start free and make it yours