When an AI Says It's Done: ThinkingBox Checks the Records

Microsoft and Hugging Face have released ThinkingBox, an evaluation tool that checks whether an AI actually finished a job by inspecting the real data it left behind. They also published ThinkingBox-Bench, a companion collection of evaluation tasks. In a joint blog post on October 3 (local time), the two companies presented results from running 507 tasks 20 times each. Instead of trusting the “done” message sent by an agentic AI that handles work on its own, they looked at what it actually changed.

It says it’s done, so why is it a failure?

A customer-service case about a delayed shipment captures the point of the evaluation well. The AI looked up the delivery status and refund policy for a $745 kitchen appliance and even created a support record. But it closed the ticket as resolved while the delivery problem was still unresolved. The evaluation required the ticket to remain on hold, awaiting resolution. Even if the AI performs every lookup it needs to, the attempt is scored as a failure if the final record it leaves behind is wrong.

The same problem showed up in tasks that appeared to wrap up smoothly. In a separate experiment comparing 12 models, 79,853 out of 121,680 valid attempts failed verification. According to the researchers, 67.24% of the failed attempts called tools that modify data and still terminated normally, with no final tool error reported either. Judging only by the progress visible on screen, it is hard to notice that the work was handled incorrectly.

Each evaluation starts fresh from the same initial state. It inspects not only the values the AI changed but also changes it omitted and changes it added unnecessarily. Of the 507 tasks, 477 are scored solely on the final state of the data, while the remaining 30 also take the content of the response into account. Whether the AI properly explained that it could not guarantee an outcome is verified with a separate question.

It works once, but will it work 20 times?

The rate of succeeding in a single attempt and the rate of succeeding in all 20 attempts looked quite different. Claude Opus 5.5 had the highest single-attempt success rate overall at 67.16%. Yet the number of tasks it completed successfully all 20 times was 241, the same as Claude Opus 5. Claude Opus 5’s single-attempt success rate was 66.50%. Even as average scores rose, the number of tasks solved repeatedly did not rise along with them.

Image: huggingface.co

Kimi-K3 succeeded at least once on 476 tasks, the most of any model. That is 93.89% of all 507. But it succeeded all 20 times on just 68 tasks, or 13.41%. Claude Opus 5 succeeded at least once on 79.09% of tasks and passed all 20 runs on 47.53%. The ability to handle many kinds of work and the ability to repeat the same work without wavering turned out to be two different things.

Cost per success: which model is cheapest?

The cost rankings also flipped depending on how success is counted. Based on usage and list prices, the researchers estimated the cost per success at $0.127 for GPT-5.6 Sol, the lowest of the group. Divide the full cost of all 20 runs by the number of tasks solved in all 20 attempts, and GPT-5.4 comes out lowest at $6.80. By the same calculation, GPT-6-Astra came to $7.45 and Claude Opus 5.5 to $7.80.

Image: huggingface.co

The ThinkingBox runner and dataset are available on Hugging Face. ThinkingBox-Bench can be hooked up to OpenEnv for evaluation. The researchers recommend checking the final record before finalizing a task, identifying errors that a retry can recover from and trying those again, and requiring human approval for changes that are hard to undo. They noted, however, that they did not measure how much these measures improve success rates.

For work that alters real records, such as refunds or support tickets, what matters to users is whether the completion notice matches what actually happened. This evaluation offers a way to check that alignment repeatedly. What remains to be seen is how much repeat success rates and costs change once error recovery and human review steps are added.