I don't think this is true! There are notable portions of the free software community that are farther left (or whatever direction you may consider it to be) than Stallman and GNU.
Debian, for instance, rejects the GFDL as a non-free license when the "invariant sections" or "cover texts" mechanisms are used. These are clauses that allow GFDL documents to include words or even entire chapters that cannot be modified or removed from copies of the document. Stallman uses this clause to include a section about the ethics of the free software movement in documents like the GCC manual. Now, explaining the ethics of the free software movement is of course an important cause. But the manner in which this is done - preventing the GCC manual, or portions of it, from being freely modified in its entirety, from being recombined freely with other documents or code with otherwise-compatible licenses - is, in the view of Debian and others, incompatible with the free software philosophy.
This is a farther / more radical view. Stallman's view is that making some unmodifiable and unremovable sections in an otherwise-free work is an acceptable compromise. Debian's view is that there is no room for compromise. And Debian is an entire (and quite influential) Linux distribution.
Or take the AGPL, for instance, sponsored by now-defunct startup Affero. There are many people (myself included) who believe the AGPL is a restriction on use, freedom 0, because it prevents you from using AGPL software in certain contexts where the requirements cannot be fulfilled. Suppose I find some AGPL code with a neat algorithm for a session cache, or something, and I want to incorporate that into a TLS library. There is absolutely no way to fulfill the AGPL's requirement to make source code available to users who interact with my code over the network. The clause triggers, because users are indeed interacting over the network, but since there's no TLS extension to offer source, I can't actually do so. This is, it seems to me, as much of a restriction on use as "You can't use this in commercial settings" or "You can't use this for military applications." There's a reason that even GNU still uses the GPL for much of its software - they know the AGPL would make lots of their code effectively unusable in contexts where it ought to be usable.
Or, in the left-libertarian direction, consider the folks who want to abolish copyright entirely. The GPL fundamentally relies on copyright law, else it cannot have its particular effect over the MIT/BSD/etc. licenses where it requires redistributing source along with binaries. The GPLv2 itself starts by saying, "To protect your rights, we need to make restrictions" - there is a coherent political view that this is contradictory. There is certainly a popular view, popular among the BSDs and others, that those licenses are freer than the GNU ones. But there is also the farther and more radical view that even the MIT and BSD licenses are a compromise, and the goal should be questioning the validity of software copyrights the same way much of the free software community questions the validity of software patents.
(In fact, among non-experts, you can find a very common view that because something is released to the public in certain ways it ought to be "public domain." The Berne rule of every creative work being automatically copyrighted isn't clearly obvious to the general public, and it's a relatively recent rule, too!)
Stallman is influential, yes, and very passionate. But that's entirely different from him being all the way at the end of the range of discourse. It is worth listening to folks whose positions are less well-known but more radical.