I've heard this argument a few times. As someone that's not new to Ruby but hasn't touched Java, can you elaborate as to what these awesome Java libraries are?
Then there's algorithms, support for just about any protocol under the sun. And the Java standard library is surprisingly complete as well - what you need might even be installed with your JVM.
Try googling for a Java library next time you think of something you want to do.
This kind of turned in to a spreadsheet gem rant, but if you step back, it's typical of a greater problem. The Ruby community is relatively small when compared with the Java community. It's also significantly less "enterprisey". This means that you'll see a lot of effort put in to things that developers like -- clever libraries, syntactic "sugar", etc -- but you're less likely to see mature libraries like Apache POI. Who the hell wants to sit around and figure out BIFF8, anyway? A lot of the enterprise stuff just isn't fun, but there are plenty of Java devs who get paid to do it for their day jobs.
By using JRuby, you can benefit from both worlds.