Google DeepMind’s XYEval Finds Relative AI Performance Falls Up to 46.7%

Google DeepMind researchers introduced XYEval, a benchmark for testing whether AI agents follow misleading user suggestions, after finding that the advice cut relative task performance by as much as 46.7%. The largest decline was recorded for Gemini 3.1 Pro on Terminal-Bench. XYEval examines the “XY problem,” in which a user focuses on a proposed solution, or X, instead of the underlying goal, or Y. The test asks whether an agent can recognize the goal and still solve the task when the instruction points it toward a misleading approach.

Source: TokenPost