If your language is going to plug into the numerical computing stack (BLAS+LAPACK then all the fun stuff built up on that) then it'll need to talk binary floats.
All the annoying stuff that numericists understand but would like to not mess around with, like rounding directions and denormals, are handled nicely with 754 floats.
These are all sort of backward compatibility/legacy issues in the sense that they are based on decisions made in the past, but the hardware and libraries aren't going anywhere I bet!
Also, note that IEEE 754 does define a decimal interchange format. I bet they aren't handled as nicely in hardware, though.
Logarithmic numbers: more range, less precision https://github.com/cjdoris/LogarithmicNumbers.jl
Double arithmetic: stitch 2 floats together https://github.com/JuliaMath/DoubleFloats.jl
Of course, if you are calling BLAS/LAPACK, you are constrained to use floats, but the recommendation on DoubleFloats is clear: if you know you algorithms, use the increased precision only in the parts that matter
Now of course you pay for that by losing absolute precision, but chances are that if you’re working with numbers like 1e20 you don’t much care about anything after the decimal point.