I went to Japan in the 90s. The amount of leftover Nazi symbols (carved everywhere, marked on city maps where old depots were, etc) and casual use of Nazi references in literature was shocking.
Any inhabitant of the temples will explain how german visitors would (infrequently) vandalize the temples. Not the intentional decoration (which is obvious) but the crude carvings into random wooden walls. I'm a little surprised that you can no longer find images of these carving after a few minutes of searching on the web. That's unfortunate, but I stand by my observations.