While it doesn't look like it's true anymore (http://www.oracle.com/technetwork/java/hotspotfaq-138619.htm...), I sure seem to remember the Hotspot JVM having an option for 32 bit pointers for ... code, I think it was. Since few apps would need more than 4 GiB of code. And as you note, as the FAQ notes, this doesn't change much for most programming.
Yeah, that sounds like it, and it's at the right time.
If your heap is under 32G Java uses pointer arithmetic to fit pointers into 32 bits. "UseCompressedOops" is your google term. The reduction in cache misses offsets a lot of the overhead.
This is also why Elasticsearch recommends using nodes with a max of 32gb RAM. [1]
That's not an option in practice, since the whole point of going 64 bit is to get larger address spaces. You could define a special purpose ABI with 32-bit pointers and 64-bit integer types (and people have done so). But doing it for the platform's standard ABI would be just commercial suicide.
I just meant long and int being 64 and 32 respectively.