Exactly. What many GoF critics seem not to understand is that Design Patterns is descriptive, not prescriptive. It's "what we've seen," not "what you should use."
They're not 'Do this, and you'll have good code', they're just an observation that good code may often use many of these patterns - not because they're forced in there, but because the solution simply maps well to that kind of model.
It's been a while since I read it, but I remember using the book itself as ammo for arguing when not to use particular patterns. They cover trade-offs, and the situations when the pattern may be appropriate or not.