Either you have a multicultural society or you have a high trust society, you can't have both. Why? Because people have an instinctual distrust of people dissimilar to them.
Japanese social groups are extremely exclusive- on the broadest level, foreigners are excluded from some establishments, banking and housing services are sometimes inaccessible to foreigners. And on a smaller level, there are also establishments that do not accept new customers without an introduction, local or not. Such exclusions are an integral part of the culture - the language itself is a shibboleth, you know your place and standing in the world by the language other people use when they speak to you.