Well Claude still thinks it shouldn't read AGENTS.md [1] so they probably also don't care about agents.txt on a web server...

[1] https://github.com/anthropics/claude-code/issues/6235