I'm not a mathematician at all (mechanical engineer), but to me, "exact" sounds like "deterministic" as an opposition to stochastic.
I though optimal conveyed the idea of "literally the best possible solution but you're still in the presence of a fully random system here".
Which might be the wrong interpretation, but hopefully it explains why some people (who aren't necessarily familiar with rigorous mathematics) use optimal.