I don't know about "many people", but I have some examples. The popular book, The Architecture of Open Source Applications absolutely refused to change its title to "The Architecture of Free and Open Source Applications" when I requested it.

There are people who think that "free software" is the loony bin and absolutely refuse to be associated to it. There are examples of that in this HN thread. There are also people who call themselves "free software" and disagree with the FSF, although they don't disagree on what "free software" means. OpenBSD is a clear example (look at their release song lyrics).