Auditor
An AI reviewer checks every answer before the customer sees it — passes, rewrites or blocks it
The Auditor is a second model that reads each draft answer of the agent before it is sent. It passes the answer, makes the agent rewrite it, or blocks it and sends a fallback message — optionally handing the chat to a human. It works in text chats only; calls are not audited.
Open it from the agent sidebar: Monitor > Auditor. Changes are saved with Save in the agent's top bar.
How a check works
- Free pre-checks run on every answer, with no AI and no cost: wrong tool arguments, a tool that failed while the agent answered as if it worked, an empty answer.
- The AI reviewer scores the answer on four criteria from 0 to 1 — Groundedness, Relevance, Tool correctness, Policy — and gives a verdict: Passed, Revised or Blocked.
- On Revised the agent rewrites the answer and the reviewer checks it again, up to the number of rewrite attempts. Serious violations are blocked at once, with no rewrite.
Settings
| Field | Default | What it does |
|---|---|---|
| On / Off | On | Turns the auditor on for this agent |
| Review criteria | — | Your extra rules: tone, language, what must not be said |
| Fallback message | built-in text | Sent instead of a blocked answer. Write your own — the built-in one is generic |
| Mode | Report only | Report only saves the verdict and always sends the answer. Fix and block lets the auditor change and stop answers |
| When to review | Every answer | Every answer or Risky only — a tool was called or the agent has a vector knowledge base |
| Rewrite attempts | 1 | 0–5. Each attempt is billed as a regular answer |
| If the answer fails | Fallback message + hand off | Fallback message, Fallback message + hand off or Say nothing |
| Timeout, ms | 8000 | 1000–60000. If the review doesn't finish in time, the next setting decides |
| If the review breaks | Send the answer as is | On timeout or a model error: Send the answer as is or Treat the answer as failed |
| Provider / Model | Same as the agent | The reviewer's model. Better a different one from the agent's |
If the order already went through, a block doesn't apply: the customer is told the order is placed and the chat goes to a human.
Start in Report only: the review runs and costs tokens either way, so watch the verdicts for a few days before letting it change answers. Until you save the screen, the agent uses the workspace-wide auditor profile if an admin set one.
Where verdicts appear
Not on the Auditor screen — in the conversation. Under each audited AI reply in Chats and History there is a verdict row; click it to open Answer audit: verdict and confidence, scores per criterion, failed criteria and pre-check failures, the reviewer's feedback, the customer's message, the base answer, every regeneration and the final answer, latency and reviewer tokens.
Cost
Reviewer tokens and every rewrite are charged to the workspace balance and shown under Auditor in billing. Pre-checks are free.
Next
- Tests — catch bad answers before they go live
- Chats — read the verdicts in context
- AI control — what happens after a handoff
Help me solve it
Happ answers right away and hands over to a manager when needed.