Go almost instant build time is a huge productivity boost when compared to Java. You get both the solidity of a typed language, and the development speed of interpreted languages (py, js).
It only ever runs tasks that actually have to be run, has integration with javac, can work in parallel, and even has cross-company build caches if needed.
Also don't forget that Java can do hot reloads with the debugger, or with tools like JRebel. Certain frameworks support it 100% and it will be much much faster than whatever go does.
Why isn't it fast out of the box? Why does the simple "provide a list of deps, provide a list of paths, build" take so darn long? Because Gradle is a great tool or something?
Don't forget that it's also already at version 8, where each change is mostly incompatible, bizarre and inexplicable breakages between versions, often due to meaningless option renames. Imagine if they spent all that effort on actually making it a great tool.