I think you really underestimate how lazy some people (like me) area. I live in New York, have no access to a garden, and the only room that gets much light in my apartment is my bedroom. Even if cannabis were legal, I would never bother to grow my own, because the plants smell quite strongly.
I also enjoy trying a wide variety strains, which wouldn't work out so well if I was cultivating only a few plants.