This will lose precision, as any floating point operation is only accurate to 1 unit in the last place (ULP). Even simple addition.
With a 64-bit mantissa, ULP(x) is ≤ 1 for x in [-2e64, 2e64]. If you have the time you can count up to 2^64 by repeatedly incrementing a float80(0.0) by 1.
Addition, subtraction and multiplication on float80s are perfectly stable provided the results stays within [-2e64, 2e64].
This will mess up all kinds of sharp comparisons on the result of casted operations.