This is rarely true. The seller has negotiated and agreed to a commission with their agent, and that is getting paid regardless of whether the buyer is represented or not. Perhaps you can negotiate a purchase price low enough that the seller's agent fee is the seller's problem, but only in a buyer's market (which currently does not exist in any major market I'm aware of). The only time it makes sense to not use an agent is if you're confident you can market and close a transaction yourself selling your own property.
It's really unfortunate that there is a 3-6% drag on real estate transactions from agent and brokerage commissions, but various attempts to disrupt this model and drive down the cost have not been successful.