OpenAI to honor shared robots.txt AI training signal

OpenAI said it will honor a shared robots.txt signal, giving website operators a way to communicate their AI training preferences to participating providers, according to a company announcement published on Fri, 30 Oct 2026. The announcement frames the change as a coordination mechanism rather than a security control. robots.txt is a public file at a site's /robots.txt path, and its User-agent, Allow and Disallow directives communicate crawler-access preferences. Under RFC 9309, those rules are not access authorization and do not replace security controls; a scraper that does not cooperate can ignore them.

Source: Blockchain.News