And it's true: the best optimising compiler in the world cannot work out the intentions you had for your code -- sometimes choosing one type of algorithm will optimise quite differently from your intention -- and if you have even basic understanding of what's happening on the hardware, you're already ahead.
Funnily enough, in my last job I met someone who had the same sort of experiences, and understanding of the actual hardware, and we lamented that no-one was growing up these days understanding the low-level stuff, incl. bus dymnamics, interaction between I/O, memory and CPU, etc, etc.
Doubly interesting was in my current job, met someone who understood JVM byte-code as well as assembler, and he claimed it was key to understanding how to get truly great performance from Java.