Yes, this is exactly what CurtHagenlocher said.
>> Most of the memory used by programs are pointers
This is a huge generalization, may I add.
Yes, I will not argue that cache usage is less efficient, but I am sure you know that register access is faster than cache access, and we double the amount of registers in x64.