It's just as likely the situation be flipped and the client refuses to pay because there's no contract. The contract is there to protect both parties.
Starting work without a contract puts you at risk as a consultant - maybe the funds haven't been approved yet and the contract will never be approved, maybe the ultimate approver will disagree with the scope and you'll go down the wrong path with your day to day contact ...
Most companies hiring contractors are going to be larger than a handful of people and it's dangerous to assume you understand the internal dynamics as an outsider.
Just like a client will check that you can uphold your end of the contract by checking your references and qualifications, it's ok to ask a smaller client like a startup for assurances that they can actually pay.