Hacker News Reader

Comment by nabla9 | Hacker News Reader

It's not about clicks or traffic. The real issue is selection effect vs. advertising effect

SEO traffic benchmarks don't measure the right ting or show real ROI. FB or Google don't want you to measure the right thing.

Traffic to the website has licks, purchases and downloads that are happening anyway (selection effect) and clicks, purchases and downloads that would not have happened without ads (advertising effect) mixed. Only way to differentiate between the two is do real experiment where you divide target groups in two sections and show only the other group the ads.

When you do this experiment correctly, you find out that online ads are usually wasted money for any established brand and lower than expected value for less known brands.

View on HN

These are pretty easy to measure on Google or FB with their "lift study" tools and a conversion pixel. The lift tools will let you create a holdout group of X% of people who should be targeted by your ad. They won't see the ad, but you can still measure the rate at which they convert on your site. Then, you can compare that conversion rate to the rate of people who actually saw/clicked the ad.

Conversion pixels are never 100% accurate (and are becoming increasingly inaccurate due to browser/OS changes) and shouldn't be used to say "This ad generated exactly X sales that wouldn't have happened otherwise!", but you can examine the relative rates from people who saw or didn't see your campaign.

I've also seen companies who don't trust FB/G to grade their own homework on a known lift study, so they measure the effect by running PSA ads. Their ad campaign will be something like "Donate to charity XYZ!" but will be set to measure purchases on their website. That lets them establish the baseline purchase rate of the customers in their audience.

>These are pretty easy to measure on Google or FB with their "lift study" tools and a conversion pixel.

I hate that Facebook and Google have normalized this invasion of privacy.

https://en.wikipedia.org/wiki/Conversion_tracking

This was normalized on the web long before these companies started - they're just the best to do it (though Google did acquire one of the earliest players in DoubleClick).

The term "conversion pixel" is a misnomer these days for a Javascript snippet or API call - the concept predates JS and used to refer to a 1x1 transparent GIF.

Anyways, that generation of tech is being pushed out due to browser privacy enhancements and will be replaced with privacy-protecting browser and server APIs that introduce noise to the data, such that you can still measure the directional/relative impact of advertising without being able to tell exactly who converted.

There are a few competing industry proposals involving differential privacy/cryptography/blockchain/fixed entropy that are being publicly discussed here: https://github.com/w3c/web-advertising

> They won't see the ad, but you can still measure the rate at which they convert on your site.

This sounds like such an assumption. There's no guarantee that any one person will ever "convert" on your site. That's such an ugly carrot that FB/G are dangling in front of you. "If you spend enough money with us, we'll eventually show your ad to someone that will absolutely buy your product. Pinky swear, just keep putting coins in the slot to continue".

Let me explain it a little differently with a hypothetical scenario.

You set up an ad campaign with an audience of 100k people who are already on your mailing list.

You set up a conversion pixel and run the ads. Let's say that Google's tools say that 10% of people who saw the ad converted. That's incredible! But considering that they were already on your mailing list, maybe they were going to buy anyways - or maybe your TV ad is actually what convinced them to buy.

You test for this by creating a holdout group from that 100k. Let's say 5k people. These are people that could have seen your ad, but we're intentionally not going to show it to them. We'll still use the conversion pixel's measurement functionality on them, and we see that 8% of those 5k people buy your product.

Now, we have a clearer picture - the Google ads drive 2% more conversions for your audience - or people on your mailing list who see an ad on Google are 25% more likely to convert than those who don't.

Nobody can predict the future, but it's a lot better to inform your predictions based on historic performance than on no data at all.

Reply on news.ycombinator.com