> If anyone doesn't meet the transaction contract (send the money or send the bitcoins), both lose the deposit.

Am I understanding this correctly that if a buyer tries to scam the seller in whatever way (i.e. will initially sign the transactions but ultimately doesn't pay) then the seller will lose his deposit? If that's the case then it won't work. I ran a P2P Bitcoin exchange and it's a pretty common thing to have "fake buyers" who place offers but ultimately don't fulfill them. With a third party you can resolve these kind of situations easily, it's much harder without them.