Japan could have just said no to the Olympics, not paid the fine and told the IOC exactly where to put the contract.
Hosting the games ends up costing the host countries more money anyway so if IOC skips over them in the future...so what?
E: this assumes the fine isn't already in escrow