Grading API and CLI

Quality checks that only exist in a dashboard get skipped. The Grading API puts the same judges behind one HTTP call, so a script, a scheduled job or a release step can grade conversations from any agent and act on the result.

What you get

• ✓Autessa exposes grading to scripts over an API key at /clients/evaluations; any enabled key of your company works.
• ✓grade handles one transcript now and returns evaluation, expectedOutcomeVerdict and passed.
• ✓jobs-create grades a batch in the background; poll jobs-get or pass a webhookUrl.
• ✓jobs-items pages through a job in submission order, optionally only one status, up to 200 per page.
• ✓items-get fetches one graded transcript by itemId or by the externalId you submitted.
• ✓jobs-list lists your company’s grading jobs, newest first, with progress.
• ✓Object and array arguments take JSON, and --json sends a whole request body.

curl -X POST https://api.autessa.com/clients/evaluations/grade \
  -H "authorization: $AUTESSA_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "agentInstructions": "You are a billing support agent. Refund duplicate charges.",
    "expectedOutcome": "The agent refunds the duplicate charge.",
    "turns": [
      { "role": "USER", "content": "I was charged twice for order 1042." },
      { "role": "ASSISTANT", "content": "I refunded the duplicate $48. It arrives in 3-5 days." }
    ]
  }'

Where the judge comes in

How the model is used here

The API calls the same model judges as Transcript Grading and the Sandbox: a quality evaluation against the agent instructions and, when given, an expected-outcome judge whose MET rating decides passed.