Having said that, the following answer relates specifically to me so take whatever I have to say with a giant grain of salt as we are probably different people. Regardless of the reasons, regardless of the back story, relationship, etc... If for any reason, I have any hesitation or doubt about who I'm working with, it stops there. That's just how I am.
There is a difference between doubt and an issue that can be resolved.