Do you give them "their" tooling? I can set up a project of the type you describe in about five seconds, because I have a template for it in my IDE. The best defaults for this that I've seen are provided by IntelliJ (do you provide this in interviews? It seems legally challenging to do) and would probably take me 5-10 minutes to navigate.
I think Java depends much more heavily on powerful tooling to do the heavy lifting, and my experience of using that tooling when I haven't had a chance to configure it in advance has been pretty miserable.