When I (caucasian) moved first to Japan for six months I had a difficult time telling Japanese people apart. By the time I had been there for 4 months this effect was significantly less. In addition I remember being startled to realize that people looked "less japanese" to me.