Fair enough, seems that it is not considered international waters, but according to UN neither Iran or Oman has the right to block traffic in the Strait:

"However, the strait is governed by international law under the United Nations Convention on the Law of the Sea (UNCLOS). This grants international vessels and aircraft the right of transit passage, meaning coastal states cannot suspend this movement, provided ships transit continuously and expeditiously for the sole purpose of normal travel."

Which makes the blockade and asking for tolls illegal under the international law, though pretty sure most countries do not care about international law at all.