This website does a really good job explaining complex calculus ideas to me. One of which is 'e', where the history and its origins will help you understand why it behaves the way it does. Without this context, 'e' is just some arbitrary number that you have to memorize. This is solely why I believe that history of math should definitely be taught.

Finance is also another area where history should be taught since ideas like continuously compounded interest was a recent development, and it's history explains why anyone wanted to compound continously.