You're right. In V8, values from a Float32Array are always converted to doubles upon access, even in cases like adding values from two Float32Arrays and storing them in another Float32Array where the result is provably the same. That costs a handful of cycles for each element.
Mozilla has a blog post about doing efficient float32 math: https://blog.mozilla.org/javascript/2013/11/07/efficient-flo...
V8 bug: https://bugs.chromium.org/p/v8/issues/detail?id=5541
Using a Float64Array is faster at the expense of memory usage. (I've never seen a case where the code generated by V8 bottlenecks on memory bandwidth fwiw.)