Devs who happily use C++ yet balk at spending a day to get a large third party library integrated into their build are a curious bunch.
Fixing a build can be pretty torturous though. My main thought when I'm looking at using a library in C++ is "oh god I hope this isn't a pain to build"
Many ++ devs, me included, come from the C background and the amount of obfuscation Boost adds to the code is simply not worth any benefits it brings along. Yes, you can do a lot with a single line of Boost'ed code, but when it maps onto pages of simpler code, you basically lose a lot of control over the code. It also bloats the binary, which again may not even register as a viable concern with some, but which does have several long-reaching complications.
This is along the lines of the traditional abstraction level debates, rather than anything particularly wrong with boost. I think it gets singled out unfairly as being a particularly difficult 3rd party library.
I should mention, people who talk without much context are a curious bunch too.
If you were adding boost to a project just to do some weird spirit/preprocessor hacks, I'd be wary. But if you told me you were going to write yet another cross platform ASIO wrapper because you couldn't figure out how to compile boost I'd be even more concerned.