Personally I think one of the hardest things to wrap your head around is how it's power-of-two fractions that can be represented exactly, not power-of-10 fractions. In other words, 0.5 can be represented exactly as IEEE floating point, but 0.1 cannot.

This is not inherent to floating point; a floating point representation that was based on decimal digits instead of binary ones would not have this issue.

It's not too hard to understand limited precision and rounding; we deal with both all the time. The part that's hard to understand is that you have to perform rounding simply to put a number like "0.1" into a floating-point value.

Specifically, in 0.1 as an IEEE float is represented as:

  val = 0x3dcccccd
  sign = 0
  significand = 4ccccd (1.6000000)
  exponent = 0x7b (-4)

So 0.1 is internally represented as 1.6000000 * 2^-4, while 0.5 has the much more understandable representation 1 * 2^-1.

This is made even more confusing by the fact that rounding performed on output will make it appear that 0.1 has been exactly represented in many cases.

So take a data type that can't exactly represent some numbers that look very simple in decimal, but that appears to represent them exactly when you print them out, but then compares not equal when you try to actually compare them. No wonder people get confused!