It has good performance in a general sense. But what it really lacks is value types and good tools for cache efficiency.
That may not be important for what you're doing. But if it is, it's a huge pain in the ass to do on the JVM. For that reason, some people want to avoid Java altogether, not because it's slow, but because it has a lower performance ceiling.
But is that an argument against Java altogether? I don't think so. If your problem is solvable with sparse FFI, that is worth doing over throwing out a massive ecosystem you're familiar with. Value Types may yet come to the JVM someday, as ridiculous as that is to say now.