The problems mentioned in that article result from software that thinks this limit is set in stone. Such software encodes data in these upper bits, and then later masks it away before accessing the pointer.
Memory addressing on 8086 inherently adds two registers every time: the pointer and a segment register (which points to a 64k region of the 1M address space with a granularity of 16 bytes). The problem is what to do when the segment plus the address overflows. On the 20 bit original implementation, you just got the rolled-over address. (What you should have gotten was a fault of some kind, of course.) On the 286, the same addition would give a valid address in the second megabyte of memory, which isn't the same memory. So the PC/AT (not the CPU) had a compatibility switch wired that would force the 21st memory address line low in real mode for compatibility with apps that actually relied on the early behavior.
The point was just to have a compatibility hack for old apps, not to pollute the architecture going forward. The reason that pollution happened was actually that the PC clone makers incompletely and incorrectly copied this feature, so later OSes never knew a priori whether or not it would be enabled by the BIOS, and they all had to have the same 10-20 instruction sequence to turn it off. And that meant that future PCs needed the same 8042-looking device on the bus to handle that request.