It only seems cheaper because rich Americans keep using their income and comparing it to the adjusted USD:YEN price. The Yen is trash compared to the dollar, so it seems affordable to an American. But it's not America.
I feel like many commenters have not been to Tokyo. Some of the apartments are literally the size of a single bedroom in an average New York City apartment.
Median household in New York is 120k; it’s 87k in Japan. Rent is over 200% higher than in Japan. Even adjusting for square footage and the earning differential, I assure you it is much easier to get an affordable place (and pay less per square foot accounting for income differentials) in Japan than in New York. Because of the Japanese transit system and the lack of household appreciation, there is a much broader range of rental prices and that range increases the further outside the city center you go.
Wanted to mention though that your sentence "the total cost of living in Tokyo blows NYC and the Bay out of the water" risks being interpreted with two very different meanings.
I've lived in 3 separate places in Tokyo (and currently own an apartment), and everything I've lived in has been bigger and cheaper than my apartment in San Francisco. I currently live in Sangenjaya (central Tokyo, a couple stops from Shibuya) and my mortgage, insurance, HOA, and total expenses are cheaper than my rent alone in SF.
My income is lower, but not in a way that I actually feel, because the overall cost of living is considerably lower in Tokyo. The conversion rate only really matters if you're going out of the country. There's been barely any inflation, so if anything, as the rate eventually goes back to normal, things will still be considerably cheaper here.
Yes, a lot of people live in tiny apartments here, but the cost of those units can be really cheap. You can pay ~$400 a month for a tiny unit here, which you simply can't get in NYC.