Probably not the guns. Germany has about a third the rate of gun ownership the US does and its rates of serious violent crimes, both with and without guns are much less than a third those of the US. All of Scandinavia, Switzerland and Iceland have high rates of gun ownership and even lower crime rates.
The prisons might have an impact, but I think at least as important is the post-release re-integration in to society. The parole system in the US is set up more to ensure that released prisoners can be easily subjected to further punishment if need be than to help them lead productive lives.
An important factor is that those countries have a distinct culture and sense of community. People are more willing to hurt others they don't see as part of a group to which they are loyal. I actually see the lack of cultural homogeneity as a strength for the US overall, but I think it tends to lead to more crime.
Another important factor is the relative lack of poverty in northwestern Europe. Someone who sees none of their prospective ways to earn a living legally leading to a comfortable lifestyle is more likely to consider illegal and antisocial approaches.