And this is the fault of the garbage collector? Doing per-primitive ops by allocating heap blocks is a disaster, sorry. No one doing HPC does that. This won't run well anywhere, though obviously a desktop machine is going to give you a lot more freedom before you see frame rate problems.
If there's Only One Way To Do It, that One Way should not cause the garbage collector to fall over and choke. It sounds like the desktop JVM handles it just fine, in comparison.
This is an area where C# is better. You can create classes or structs and structs go on the stack. Structs are a lot more appropriate for objects like rects.
Make sure you release the instances back to the pool on every exit path, but don't release them if you let a reference escape into another scope.
Have fun debugging the issues that crop up when you accidentally release an instance that's still being used back into the pool.
Have fun hunting down that one code path that isn't returning instances to the pool, and as a result is draining the pool down to 0 and causing you to gain nothing from it.