But people, including advocates and reporters, discovered the directives in robots.txt and <meta> tags, which have been out-in-plain-sight for at least a decade [0], and that has apparently led former employees to speak out and/or leak internal documents that will very likely lead to an investigation. Tax prep companies have spent millions per year in lobbying, and now a major, easy legislative victory may be spoiled because of robots.txt. It's pretty rare that we see grass-roots complaints actually stymie a popular bipartisan bill supported by industry lobbyists, nevermind because of a webdev technical issue.

[0] http://web.archive.org/web/20080512004753/http://turbotax.in...