Do you use this approach?
If so, are you willing to share an example robots.txt file? What bots would be allowed? Google, Bing, ...? Those?
If so, are you willing to share an example robots.txt file? What bots would be allowed? Google, Bing, ...? Those?
Any robots.txt I might suggest wouldn't necessarily be appropriate for anyone else. It's a policy that site stakeholders need to decide for themselves I'd say.