GPT-4o scored 0.82 on non-public books—what does it mean for OpenAI’s GPT training data?
Photo: The Cryptonomist
A study of OpenAI’s GPT training data found a marked difference between models tested on copyrighted, non-public books: GPT-4o scored well above random chance, while two other models performed close to chance. Researcher Ilan Strauss used the results to examine potential access-rights violations, not simply whether material appeared in training. The study, titled Beyond Public Access in LLM Pre-Training Data , used a legally acquired collection of 34 O’Reilly Media books containing 13,962 paragraphs . Strauss applied the DE-COP membership inference attack to test recognition of paywalled material that, the study says, the models lacked permission to use for training.
Source: The Cryptonomist
