You're right that that is a theoretical possibility, but I think it's very unlikely in practice. In the VC world, term sheets are only technically non-binding; breaking a signed term sheet is considered a Big Deal.
What I have seen - and just once - is that a DD did in fact turn up some major issues, the investors backed out and the startup then made a stink pretending they were stiffed. But the truth on that one will likely never see the light. So not 100% of these can be laid at the door of the VCs either.
I've also had founders back out of accepted term sheets, which is also a pretty shitty thing to do.