I've lost count of the number of times runtime behavior and performance issues have been traced back to (for us) far-from-optimal design choices in third-party libraries that probably seemed like reasonable choices to the library authors. Particularly in open source, you see remarkably little investment in performance engineering or scalability, so it is always straightforward to write your own implementation that is many times faster on the same hardware. Even basic things like popular data format conversion libraries are surprisingly consistently suboptimal. In most cases, the kinds of changes we'd want to make cannot be upstreamed to that library; most open source libraries heavily optimize for simplicity and the widest set of users to the exclusion of maximum performance. In an age when hardware can deliver >100 GB/s of throughput, many popular software libraries deliver <1 GB/s per core even though far better can be achieved with a modicum of diligence.
Another issue is the impracticality of threading fine-grained metrics and observability through a hodgepodge of libraries that were never designed to be used that way or to work together. This is a big deal if you want to heavily automate operations.
There is also a real issue with supply chain risk in terms of both quality and security. The vast majority of open source libraries are implemented to a lower standard of quality and security than the rest of the code base. And those libraries are often not really designed to be testable to the extent that would give similar confidence in the implementation quality. Combine this with the reality that the software needs to be deployable in environments that require higher than average assurance makes this a business risk.
As for the manpower cost, it has been low in practice. An engineer-month here, a few engineer-months there, with the benefit of better performance, more optionality, and much easier maintainability. In most cases, we would only use a tiny subset of library functionality regardless, so it adds a large surface area for a narrow benefit. Much of this code is also reusable. The existing open source libraries are very useful here because they provide something to measure against.
I also like that this forces a discussion of actual requirements for that functionality. Far too much software makes their requirements whatever a library offers.
I arrived at this point gradually over many, many years. The practical maintainability and opportunity for optimization have been key. In far too many cases, it turns out that a library dependency could be replaced with a couple hundred lines of code that actually worked better and which matched the design and style of the rest of the code base. There is a learning curve but once you've written a common class of library once, it is straightforward to do it again.