Implementations like SBCL have a similar overhead for a function call containing arithmetic. However, if you're doing more arithmetic within the function, the bit shifts etc. are only done once as a part of the function's pre/postamble. In fact, fully unboxed 64-bit arithmetic with integers or floats can be done if these values aren't crossing function boundaries or into the garbage-collected heap. (The latter, a memory read/write, is way more costly than a bit shift anyway.)
The term of art is "inlining" or "open-coding" arithmetic. Raw arithmetic can be open-coded, but data formats must be respected across function call boundaries and in the heap. Therefore, if you inline the function containing the arithmetic, you eliminate the pre/postamble overhead too. This is the same as in C-like languages as well, it's just that their pre/postambles are different.
"Block" or "whole-program compilation" can also sometimes eliminate these extra instructions, at some modest expenses (compile wall clock time, implementation complexity, increased size of compilation units, multiple function entry points, modularity, etc.).
OCaml's built-in Flambda [1,2] is a project/technology that will automatically leverage some of the above techniques (especially inlining) to achieve higher performance.