Without the exchanges, buying and selling for 'real' currency would be next to impossible.
Without access to 'real' money, the get rich quick crowd (and probably a lot of the scam crowd, and probably a lot of the dark market crowd) move on. With them gone... there's probably not very much left of a cryptocurrency user base.