We don't put people in jail for pot, at least not here in California. We don't have the room. Officially in this state 0.3% of inmates are in for "possession for sale" and "sales" of marijuana. They don't even list simple possession.

http://www.cdcr.ca.gov/Reports_Research/Offender_Information...

It's true federal penitentiaries have a larger percentage of inmates in for simple possession, but the federal prison population is a tiny fraction of the number of people incarcerated by the states.

And while I don't have any evidence to back it up, the comparatively small number of people in federal prison for MJ possession suggests they're either the most unlucky people on the planet or they're really there for some other reason even if the government could only make the drug charge stick. Think Al Capone and tax evasion.

tl;dr: Legalizing pot isn't going to affect the prison population very much.