That’s a bit hyperbolic. How these issues are handled vary by each country and are agreed upon by the host country. Here is a link on the topic.
https://www.findlaw.com/military/criminal-law/foreign-crimin...
https://www.findlaw.com/military/criminal-law/foreign-crimin...