Preferably a permanent solution is one that other such calamities could responsibly use as well without eventually ruining the world.
The impact of tossing it into the ocean is negligible , but
1) it sets a precedent unless the rest of the world reacts in a negative way and punishes the guilty (of which it seems TEPCO has plenty)
2) the negligibility of such action is rendered moot if the precedent set allows for such action to be considered a viable clean-up method in the future.
"A good permanent solution" should be one that can be enacted safely by future people confronted by similar issues.
'Out of sight, out of mind' has never been a responsible tactic, look at the ongoing legal struggle between countries behind the cleanup of the Ganges, for one example.
I don't have a better alternative, but I don't like the idea that system operators in the future may now view this kind of action as appropriate anywhere but the most dire of situations.