> What happens if you have a string associated with an object that you don't want to cache miss on?
Then you embed the array of chars in the object, just like in C++. I don't see what having a GC has to do with it. As to embedding arrays in objects in Java, I think it's planned as part of the value-type project, but no prototype yet. I know John Rose (JVM architect) said that the two features they intend to address are arrays of structs and structs of arrays (but not buffers of structs of arrays).
> you'll pay the int->float cast costs for one
HotSpot should optimize it away (not sure about byte buffer usage, though). It sees an integer read and a cast to float, and reads directly into a floating-point register.
> Java doesn't have a nice way to wrap an immutable string
You mean mutable string? Well, there's the CharSequence interface that String and the mutable StringBuilder implement, but it's true that not many standard APIs accept a CharSequence.
> I think it's important to understand they we're talking about a specific set of areas that C++ excels in.
I agree. C/C++ is unparalleled for single-threaded traversals over encoded binary data (e.g. video encoding/decoding and other signal processing) and for performance-sensitive applications running in memory constrained environments . Managed runtimes even try to compete in those domains.
In other cases, the extra performance you can squeeze out is too insignificant (and growing less significant with time) to justify the extra effort. And in some cases of growing importance (interesting multi-threading), C++ has to work really hard to even match GC-runtime environments. The real cost that is always paid is the significant RAM overhead.
And BTW: Android isn't Java. It's compatibility with Java has always been very partial, and, of course, its development is not directed by the JCP.