True, though I'm undecided on whether that justifies tracking clicks. The fundamental problem is that neither that advertisers nor the websites trust each other. If the advertisers trusted the website, then the Referer header would be sufficient for the advertiser to know where clicks came from, and the website host could verify that those are correlated with their own server logs.
However, this would require a cost, in terms of refusing to do business with untrustworthy websites. Rather than accepting that as a fact of the business, websites/advertisers instead push the cost off to users as an externality. This cost is paid by the user in time/bandwidth (larger page load times) and privacy (ad trackers), even though they were by no means the source of the problem.