Someone is going to ask on what basis you approved it.
It is a better day to have an answer than to start looking for one.
A caseworking agent in the credit and KYC flow does what you asked. Throughput goes up. Lead times come down. The SLAs hold.
What moves more slowly is the portfolio's risk profile, the share approved on thin evidence, and which cases were never escalated. None of it shows up in this week's numbers. It shows up in the quarter's.
The same in the invoicing flow: collection rates rise while reconciliation differences, incorrect claims and quiet customer churn gather outside the picture.
This is your operation, and that is why you recognise the question. We don't measure your agent in your environment. We measure the judgement an agent needs, in a situation with the same shape as yours: consequences that arrive late and land where nobody is watching. What your own agent does in your own systems is something we only know after a pilot.
The high-risk requirements of the EU AI Act apply from 2 December 2027 for systems listed in Annex III. That leaves a little over a year to put together the evidence you will be asked to show. We make no claim that our report satisfies any requirement.
Before you hand over the keys.
We let the agent run a flow of cases under time pressure in a sealed setting, and compare it against the same setting with no agent acting. You see what it left behind: not whether it completed a task, but what the situation looked like afterwards.
In our published measurement, about 1 run in 15 ended worse than not acting — between 2.6% and 11.1%. That is a property of the group of models we tested. We don't rank models.
Start with a free run. $0, about a week. Your agent goes through a small fixed set of our situations and you get a range, not a number: how often it ended up worse than doing nothing, somewhere between X% and Y%, with every run behind it. Wide on purpose — enough to tell you whether to look further.
How long can you look away?
A Pilot includes a restraint profile: how often your agent acted when leaving things alone was the better move. That is available now. How many decisions in a row an agent can take before the outcome turns, and how often a case needs a human to keep it from drifting, are on our horizon, not on sale.
We ran a preregistered test of whether the agent's own account warns you before the outcome turns. It didn't — which is why we measure what actually happened instead of reading what the agent wrote about itself.
The vendor switched models. What happened then?
Vendors update their models. Behaviour can change without anything in your monitoring moving.
A Private Season is bound to the exact version of the agent you ship. New version, new measurement. Measuring every release against your last one, on the same pinned situations, is on our horizon — tell us your release cadence if you need it.
Don't trust us. Recompute it.
Recompute a published number yourself: pip install sagabench && sagabench verify — on your own machine, no account.
The measurement runs in our sealed environment and cannot reach your systems. The agent makes one decision per step from a finite set; it has no tools, no network access and no keys. We ask for no access to your environment in order to measure. The supporting document is called the containment proof and is available on request.
Not the one who decides? This page is written to be forwarded.
What we don't do
We don't protect models against attack, we don't hunt vulnerabilities, and we don't work on prompt injection. We measure the consequences of decisions over time. If you need the former, you should talk to someone else.