Another good one - back around the era of the original NVIDIA Ion boards, I was helping to run a cluster of these boards as an experiment in low-power computing. [2]
Some ran Linux, some ran Windows. Running CUDA code under Linux headless is fine, running it under Windows with a non-Tesla GPU was nontrivial at best (and involved hacking up the Tesla variant of the driver to add some PCI IDs). Unfortunately, it turns out that this breaks if you don't have an actual display attached to the machine.
The solution that was implemented was to take 36 naked male VGA headers and solder resistors across just enough pins to convince the system that there was a display there, and then install them.
Or the Samsung SMART IDENTIFY hard drive bug - which meant that the advice "disable SMART to keep your data safe" was sometimes valid. (The drives had a FW bug that caused them to drop data in the write cache if they got a SMART command before flushing it.) [3]
I'm sure I'll think of more later.
[1] - http://forums.steampowered.com/forums/showthread.php?t=10931...
[2] - http://www.nvidia.com/content/gtc/documents/sc09_szalay.pdf
[3] - https://www.smartmontools.org/wiki/SamsungF4EGBadBlocks