> You may have a lot of things you do not want the world to know about. (Eg user uploaded media). Use robots.txt to hide them from search engines.
That's not what a robots.txt file is for. It doesn't hide files, if anything it makes files that you don't want the world to know about easier to find.