I've spent a lot of time doing Rails and Go, and I also find that on balance, it doesn't make that much difference. It's hard to measure though.
I'm not "against" frameworks, but there are up- and downsides. E.g. in Rails I had quite some trouble with complex forms, which eventually resulted in form_this[1], which actually worked pretty well for the application I was working on. For a lot of apps you don't need this kind of thing, but when you do, it's quite difficult to do something outside the Rails "happy path". Maybe newer versions of Rails handle this better; I haven't really kept up.
I found the Rails codebase rather tricky at times. You get things like:
[:method1, :method2, :method3].each do |name|
define_method name, [..]
define_method "#{name}=", [..]
define_method "#{name}_foo", [..]
end
And it's really neat you can do that in Ruby, but it can also make a codebase kind of tricky to debug – it's the old "you need to be twice as clever to debug code than to write it". While I generally found Rails to be pretty good, it
does have bugs and quirks.
On the flipside with Go you spend a bit more time writing your own code, but that code will be better suited to what you're doing. Sometimes people seem to be afraid to write code, but I found it's not a bad thing and "do it yourself" can be quicker and better than "adapt existing thing to your needs".
On balance, it seems to more or less cancel out. Maybe there's some difference one way or the other, but it doesn't seem to be huge.
[1]: https://github.com/bluerail/form_this