How do you define "users"? Say something you ordered or bought in a store was shipped along that road, did you "use" it? If one of the employees of that company had to use that road to get to work, did you "use" it? one of their family members, without whose job they wouldn't have theirs?

Should the owner of a toll road have to pay money back to subsidize the costs of the roads which give access to their toll road? Vehicles need to be able to get on/off the road before tolls can be collected.

Or, if we apply your logic, then it is impossible for people to live in rural areas, because the cost of the roads is more than the local economy can bear.

Road networks are an infrastructure investment. Since they are networks, the value comes from the entire network. What is the value of a road with no on or off ramps?

Because the value is so diffisive over both space and time, the only effective way to account for and recoup the cost is through a mechanism that can take a share of the economic growth resulting from the network.

i.e. taxes.