> Including walking to/from stations and bus stops, taking the bus/train costs me around 80 minutes of my day each way
That's not actually how we define "amazing public transportation network". My experience of public transit in Seattle is "about 5 minutes away from a bus stop, anywhere, and about 30 minutes to anywhere I care to go in the city, and maybe an hour if I go across the lake to Bellevue or Redmond".
The fact that it takes you longer for traveling within the city than it does for me to change cities suggests to me that Tokyo's public transit is either ridiculously bad or you have an extremely unusual case.