Inevitably. Which means you'll have to resort to "real programming" (as opposed to component assembly) for /that part/ of your project. I don't see this as a fundamental reason why component-assembly can never become viable. As we get better at creating flexible components, these situations will get less common, but they will never go away.
> you'll run into a requirement that's simpler to implement directly than it is to write the glue code for all the components you could use to solve it
100% this. For me this pretty much sums up why component-assembly isn't viable today. For all but the simplest components this turns out to be the case. (e.g. date-picker, file-uploader, or maybe something a bit bigger with /very/ fixed requirements, like a disqus comment trail). But jumping from this to "I don't think it's every going to happen" is overly pessimistic. The glue code is too hard to write? We need a better way to write glue code. That could be a fundamentally different type of language, or a fundamentally different conception of what we mean by "component".
(Aside: In my foolishness I am working on such things).