If your heap is under 32G Java uses pointer arithmetic to fit pointers into 32 bits. "UseCompressedOops" is your google term. The reduction in cache misses offsets a lot of the overhead.
[1] https://www.elastic.co/guide/en/elasticsearch/guide/current/...