So this exploits the fact that floating point numbers have finite precision (and perhaps uneven spacing) to generate non-linear operations?
That's actually a really cool usage of the specification!
That's actually a really cool usage of the specification!
TensorFlow by default is built with denormals off (ftz=true), so denormals aren't relevant for the applications we're interested in. We have updated the post to indicate this — thanks for the feedback!