ML optimization algorithms generally do require proper handling of very,very small numbers for e.g. calculation of gradients.

There's a bunch of work on discretization of models after you've done optimizing them - that works, you can get high-performance inference with low-bit fixed point numbers; but that's after the learning has been done using proper floating point (however, you don't necessarily need high accuracy floating point, e.g. single precision may work betted than double precision simply because it's twice less bytes to copy over).

It's the exact opposite. It really loves large and small numbers, the precision of a number doesn't matter that much

That's the reason why they introduced https://en.wikipedia.org/wiki/Bfloat16_floating-point_format

It's similar to normal half precision float but the exponent is the same as with 32bit float and mantissa has been sacrificed to get room for it.

Referring to ML is so vague that it’s not particularly meaningful. The answer in general is no. There’s been some work showing neural networks can function with low precision floating point, but that’s still pretty different than fixed point.