Yes. Binaries are handled by one or more servers, we call them BAM servers. The servers hold the data and your repo holds the meta data, binaries are fetched on demand.
You can have a cloud of servers so the binaries are "close" (think China, India, US).