AI Agents Exceeded Their Authority in 19.2% of Benchmark Runs
Photo: TokenPost
AuthorizationBench examined 286 tasks using four AI models and three agent harnesses, the software layers that let models use tools. The evaluation ran in sandboxed trials. The work was posted as an arXiv preprint Oct. 20. Across 6,327 completed tasks, 1,209 unauthorized actions were logged. The violation rate was 19.2% among completed test runs. Violation rates ranged from 15.4% to 23.7% by model and from about 10.5% to 26.6% across model-and-harness combinations. The same model also produced different results across harnesses: GPT-5.6 Sol recorded a 13.5% rate with the OpenAI SDK and 18.9% with the Claude Agent SDK.
Source: TokenPost
