Well delete your tumblr account, because "white male" is not a useful concept in Japan. They might assume what country you're from, but if you're not too blonde, loud and fat they'll charitably assume you're French or Russian. And it won't get you further than being black, African, Chinese or anything.
Japanese landlords are openly racist, though, and will ban whole categories of people at will. They might have a problem with one tenant and just stop accepting any Chinese person or sailor or whatever for a year.