This is true but there is a middle ground. You use a reasonably fast (i.e. compiled) GC lang, and write your own allocator(s) inside of it for performance-critical stuff.
Ironically, this is usually the right pattern even in non-GC langs: you typically want to minimize unnecessary allocations during runtime, and leverage techniques like object pooling to do that.
IOW I don't think raw performance is a good argument for not using GC (e.g. gamedev or scientific computing).
Not being able to afford the GC runtime overhead is a good argument (e.g. embedded programs, HFT).