LLM Sampling Bias Raises Simulated Security Risk in Patrol Test
Photo: TokenPost
Tests of specific language models found that uneven sampling raised a simulated attacker’s success rate in a four-server patrol task to as high as 99.9%, underscoring a security risk when agents must make randomized choices. The paper, “The Illusion of Stochasticity in LLMs,” tested requests for uniform discrete, uniform continuous and Gaussian samples. Each configuration used 1,024 outputs. In a representative table covering four models and three distributions, all 12 goodness-of-fit p-values fell below 0.05, providing evidence against the requested distributions under the tests used. The tests used chi-square comparisons for discrete uniform sampling and Kolmogorov-Smirnov tests for continuous distributions.
Source: TokenPost
