For the same reason we had a toilet paper shortage: there are two legal "types" of the thing and legally you can't "cross the streams". Not without the exact same people complaining to make it housing crying bloody hell about the difference.
These are also the NIMBYs who block EVERYTHING in SF Bay Area.
I know this is an unpopular opinion, but I think one of the strongest opponents of housing is NIMBY. If you have an office building that could be turned into housing, I think you’re going to see outrage from the neighborhood that it’s going to become a slum, or some kind of crime infested drug riddled cesspool of desperate people.