It's probably not as romanticizing as you expect.

Watch the beginning of Totoro to get an idea of what an abandoned house in rural Japan looks like (rundown, in the middle of nowhere). Vacation homes anywhere get quite dusty in a mere week, so you'd be looking at a spring cleaning event on every trip (very much like in the movie).

Bear in mind also that flight tickets to Japan aren't exactly cheap either; you can get a week worth of hotel stays in the US for the price of the round trip flight from US to Japan.