In the United States, buyers don't directly pay the realtor commission. The seller generally pays ~6%, but sometimes that's negotiated down. This commission is split between the buyer's agent and the seller's agent, with their brokerage typically taking a percentage. But a buyer wants to use an agent because their perception is that they're not paying for it, and most buyers aren't well versed with the house transaction process, which is complex.
You could certainly make the case that a transaction without agents would mean either more profit for the seller, or a lower house price for the buyer, but I think it will take many years until that is the norm. There are a huge number of laws built around real estate transactions, and the NAR is a major major lobbying entity.