This would seem to be only a temporary local solution. Once world population stabilizes or starts shrinking, you can only increase population by exporting the problem to somebody else.
"Good news"! Due to the climate crisis, large sections of the globe are likely to become uninhabitable, so even if world population declines, there will be an influx of refugees into habitable locations. In other words, local population density could easily increase worldwide even if there are fewer people overall.
Or Japan could itself sink into the sea and skip the step of moving people into the newly depopulated islands.