Sometimes. I think the best response in theme of the thread so far is "When it starts to matter, I'll spend the time looking into it. Until then, I try to know just enough to clue me into whether it might matter."
As a concrete example, I know why stack and heap matter for performance, and I know that maximizing computation in a hot loop in something like assembly requires a fair bit of knowledge of the architecture and what instructions utilize what resources in a CPU you you can plan out usage optimally, but beyond reading some interesting articles that have popped up here, that's all I've bothered to internalize. I haven't done almost anything in C or C++ since college well over a decade ago, and all my day-to-day work is done in Perl. At the point I'm in need of a customized very optimized solution for a computationally heavy workload, I have promising avenues of inquiry to start with. The way I see it, that the optimal strategy for my current goals, and it's been the optimal strategy for quite a while.