I would simply state I can't sign as is. The only times I got pushback (rare) an annotated letter from accountant or PCG (now IPSE I think) contract review and the agency buckled. I also demanded proof that the agency-client contract matched on IR35 terms. Was never once caught by it.
Does the agent want to lose commission? Does the agency want me telling the client direct why I've changed my acceptance?
I realise regulation and the market has changed some since I last contracted, hmm, 8 or 9 years ago. Still can't imagine signing any agency contract as-is, they were always horribly one-sided affairs.