I've used http://www.dx.com quite a bit. Prices are really, really low. Shipping is free. However, you may have to wait two months for your products to arrive.

Since most of this stuff is based off of open source hardware it's pretty much identical to the name brand stuff. So it pretty much comes down to a cost/time ratio consideration when first starting out. Before you know it though, you will have quite a large collection of parts built up.

Also remember you can hit garage sales, recycle yards, and thrift shops to find a lot of old hardware that you can salvage valuable parts out of for less than the price of the parts themselves.

Edit: made link clickable