tympious

I mean, "allow" or "regulate" don't _really_ apply here - there was never any enforcement regime around robots.txt, just a convention based on the general expectation that you don't claim ownership of whatever passes your line of sight.

Taking either step you suggest (along with robots.txt or eqiv.), it would seem fair to expect that Brave, Bing, whomever, would not feel it their neutral/natural domain to include in a public index.

What if I consider (some or any of) my ideas to be un-indexable, not directly suitable to representation in any hierarchy other than those I may set them in?

"The Universe is a hologram: Stephen Hawking's final theory, explained by his closest collaborator"

[edited: deletions]