When you hit a critical size your database can no longer handle the throughput of the lookups. You'll then install memcache or some other tech and the stack JUST FOR handling the tokens is a majority of your datacenter load.

Most startups and projects never hit this size, they usually fold before that level of growth. It is much lower than one would assume though since every request made to an API has to do the lookups etc.