I don't see that as a problem at all, just like I don't see header files in C++ as a major problem, there are benefits as well and Java has the best IDEs of any language I've worked in except maybe SmallTalk.
It's a stupidly simple language.
Just because people actually use it and thus IDEs were developed to aid the development process doesn't make it a necessity at all. I have programmed Java countless times from vim without any plugins. The only pain point is imports, which would be similarly painful in any other language ever created.