At this point, I really couldn't back out if this startup even if I wanted to (and I have no desire to): I quit my job, spent a ton of money to move, and told everyone I know about my plans. As Dharmesh Shah of onstartups would say, I'm emotionally committed, which is more significant than my financial commitment (He has a good post about this at http://onstartups.com/home/tabid/3339/bid/1204/Startup-Founders-The-Involved-vs-The-Committed.aspx ).
As dfranke pointed out, we have more options than only accepting YC funding or not accepting. The real question is whether it makes more sense for us to keep going with money we have in Denver and find angel funding on our own, or move and take YC funding in a few months.