From 2000 (gasp), here's an article from Jacob Nielsen on why you only need to test with 5 users: http://www.useit.com/alertbox/20000319.html

More recently, here's a related article on Multiple-user Simultaneous Testing, when and why you'd test with more people and how to actually do that: http://www.useit.com/alertbox/multiple-user-testing.html