#AI scraping

• OpenAI Now Honors robots.txt AI Opt-Out, But Only for Nice Bots

  OpenAI, Google, Microsoft and Anthropic have agreed to honor a shared robots.txt opt-out signal for AI training. It's a practical win for site owners, but only against bots that…