Deconstructing the Google Analytics tracking script | Hacker News Reader

Deconstructing the Google Analytics tracking script

billfranklin.svbtle.com

217 pointsBillFranklin81 comments

Documented, including the official unminified source, in Google's analytics.js reference: https://developers.google.com/analytics/devguides/collection...

Would be nice if they included what variable names they use inside google instead of just i s o g r a m.

They explain what the variables mean in the function header block;

   * @param {Window}        i The global context object.
   * @param {HTMLDocument}  s The DOM document object.
   * @param {string}        o Must be 'script'.
   * @param {string}        g Protocol relative URL of the analytics.js script.
   * @param {string}        r Global name of analytics object. Defaults to 'ga'.
   * @param {HTMLElement}   a Async script tag.
   * @param {HTMLElement}   m First script tag in document.

It might be easier to read the code if they used different names, but any good refactoring tool will replace them with your own names without any problems.

I've noticed several minified JavaScripts that have the variables spell words like the i s o g r a m here. Haven't bookmarked any so I can't find an immediate example. I thought it was just a curiosity, but I think the frequency is increasing. I wonder why. Just for giggles? Or is there any particular reason?

kanyewest.com replaces the letters with "k,a,n,y,e"

    !function(k,a,n,y,e){k.GoogleAnalyticsObject=n;k[n]||(k[n]=function(){
    (k[n].q=k[n].q||[]).push(arguments)});k[n].l=+new Date;y=a.createElement('script');
    e=a.scripts[0];y.src='//www.google-analytics.com/analytics.js';
    e.parentNode.insertBefore(y,e)}(window,document,'ga');

    ga('create', 'UA-34495711-11', 'auto');
    ga('send', 'pageview');

that's hilarious, I wonder who makes/manages celebrity sites like that.

Lane Goldberg in this case.

http://www.builtbylane.com/

Using 1-character variable names saves a small amount of network bandwidth on a massive scale (for every site that uses Google Analytics). You then might as well make the 1-character variable names spell out a word, and that word has to be an "isogram" in order to provide unique variable names.

Here is an example for you:

   (function(h,o,t,j,a,r){
        h.hj=h.hj||function(){(h.hj.q=h.hj.q||[]).push(arguments)};
        h._hjSettings={hjid:1,hjsv:5};
        a=o.getElementsByTagName('head')[0];
        r=o.createElement('script');r.async=1;
        r.src=t+h._hjSettings.hjid+j+h._hjSettings.hjsv;
        a.appendChild(r);
    })(window,document,'//static.hotjar.com/c/hotjar-','.js?sv=');

https://docs.hotjar.com/v1.0/docs/understanding-the-tracking...

I believe that Google set a trend, 'HotJar' is a newbie on the block. If anyone can find an example that predates 'urchin.js' (or wherever this started) please share!

I don't know where it started for software, but I can give you an example that predates the web.

In the 80s and early 90s my mother was an interior designer at a high end furniture store. There were no prices on the floor models because most things could be customized (fabric, arm shape, wood, etc) before delivery. However there was a base price that the designer could use to (I guess) keep people within their budget.

They used a ten-letter isogram to print the base price covertly on the furniture labels so that associates could figure it out without going to the back office and looking it up in the computer.

The last word I remember them using was Cumberland. C=0, d=9. So if the base price for a dining room table was $15,840 the tag would have urnec written on it somewhere.

Used car dealers do that. There's usually an alphanumeric that tells the salesperson how much they paid for the car and some other stuff.

that's the kinds of insightful comments I come here for. Side note, I remember in a gift / greeting card shop growing up, everything had long numbers printed on a little sticker on the back, turns out the price was just the last 4 numbers (divided by 1000).

You can do it too: https://github.com/shinnn/isogram

Web version: https://isogrammer.com/

Increased FUD compared to using the plain alphabet maybe?

More in the spirit of an easter egg than FUD, I think.

Who says there are different variable names used inside Google? This is a single-function code snippet that is entirely public.

This is just an analysis of the snippet, not the script it loads (www.google-analytics.com/analytics.js).

If there is interest, I could analyze that. (I'm not the author of the article.)

I'm interested. I built something to track events for analytics long ago ~7 years. Constantly surprised it's stood the test of time. Coincidentally the loading portion is similar to the GA loading portion analyzed in the article.

Lots of questions, I would love to understand how GA are capturing events that are sometimes trapped and not allowed to bubble, etc. Are they capturing events at document.body level, etc? What type of IDs do they use to bring sessions together, do the ID events similarly? Do they queue up events to batch them or do they trickle them in. How do they deliver events when unloading a page, etc.

Google Analytics does not capture any DOM events. Data gets into the "Events" report in the GA interface when a developer explicitly makes a function call like `ga('send', 'event', Category, Action, Label);`. Google Tag Manager can automatically add tracking to several types of DOM interactions. GTM captures events at the document.body, and you would generally need a work-around if something is screwing with event bubbling.

Nowadays, connecting Sessions and Users are handled by storing an ID in a first-party cookie named _ga. There is no such thing as an Event ID in Google Analytics, unless you implement one manually in a Custom Dimension. The Event report has Total Event and Unique Event metrics, and it's up to the user to choose whether they want to pay attention to deduplicated Events or not.

On the web, hits are not batched, they are handled as they come in (except for a throttling limit). The Android and iOS SDK batch hits by default.

On pageload, queued hits can be sent with the "beacon" protocol, which is a HTML standard that exists specifically to solve this exact problem. In browsers that do not use the beacon protocol, the hit is simply dropped.

Any reference to the beacon protocol? I've never seen or heard of this. I've implemented mechanisms to periodically drain queues and clean up onbeforeunload for single page apps. Does GA's event queue span pages?

See here: https://developer.mozilla.org/en-US/docs/Web/API/Navigator/s...

The "protocol" is just a deferred HTTP POST request that does not block page unload. Please use this as it is widely supported and provides better UX (page is not blocked by sync XHR).

Google Analytics documentation on transport method: https://developers.google.com/analytics/devguides/collection...

GA does not store any data that persists from page to page other than the client ID cookie.

I've reverse-engineered most of their minified code, and it's a huge pain. Since they use closure compiler, there's a huge object pool that gets referenced for every single thing. It's supposed to make the source a little smaller, but geez.

Unless I'm misremembering, something that I found interesting was that the cookie wasn't randomized, it was a fingerprint based on a large number of params.

If you could analyze the old ga.js and figure out a way to accurately duplicate the way the old utmz cookie was created, you could probably make some money off your effort. I've yet to see a great solution to this.

How old is "old", and what do you need it for? I've written code to parse the utmz cookie and separate code to recreate the whole organic/referral/utm attribution model in GA using first party log data. The only major drawback AFAIK is that adwords utm parameters are obfuscated and Google doesn't provide a way to resolve glclids to their campaign's utm params.

it's dated, but still the best public resource on de-obfuscating gclid's.

https://deedpolloffice.com/blog/articles/decoding-gclid-para...

There's no magic to tracking cookies, unless you don't care about regulation, why not use your own tracking cookies for this?

Sure, there is interest!

The reason why the arguments spell isogram is because you can't have two arguments with the same letter. So they chose the most meta isogram, "isogram." Surprised the author missed that and assumed it was just reference to the script itself.

Thanks for clarification and also thanks for the explanation of an isogram. Or the nudge in that direction.

Something that has always puzzled me is why only GA follows the saner approach to push all the function calls in an array which the async loaded script can pick up later. Many of the popular analytics solutions (like, Segment, Mixpanel) create a factory function that initialises all API calls with a generic body. It seems rather unnecessary and only adds to the boilerplate. Take a look at the GA's init code and compare it with Segment's [1].

[1]: https://prnt.sc/fpj9ec

Some libraries (fb, ga) using a single factory method that handles all commands. Ex: fb('init', {})

The others (segment, heap) use explicit functions for different actions. Ex: heap.init()

The main benefit is you can detect API violations without loading the async script. If you call an invalid function with the former you have to wait for it to execute at some point in the future.

This allows you to surface errors immediately - breaking current exection - even if it's additional libraries havent loaded.

Having written one of those analytics snippets... Hindsight is 20:20. When I wrote it, having a facade class seemed reasonable. Eventually I realized the GA approach would've been better, but at that point we couldn't change it because we were worried it would freak out our customers.

Anyone not wanting to use the JS might be interested to know that the Google Analytics Measurement Protocol is fully documented, and you can create your own front-end implementation, should you wish:

https://developers.google.com/analytics/devguides/collection...

One thing to be aware of is that latent hits, via the Queue Time "qt" paramter, have a maximum delta of 4 hours:

"Used to collect offline / latent hits. The value represents the time delta (in milliseconds) between when the hit being reported occurred and the time the hit was sent. The value must be greater than or equal to 0. Values greater than four hours may lead to hits not being processed."

Would be nice if that delta time could be greater than 24 hours, eg. a human approves that a low-volume contact form submission isn't spam and that hit is tied to a GA goal.

Or server-side. We used it to provide quick insight into API useage with a tool the client was already using

This is awesome, it includes IP override and what not. I wonder why more people don't invoke this information server side during their own requests instead of loading the third party JS on the frontend.

I wrote almost the same post, with almost the same title a couple of years ago - https://www.ignoredbydinosaurs.com/posts/239-deconstructing-...

I like mine better :)

It is pretty obvious that the snippet loads the real script. I expected this article to be about what the actual google analytics tracking script does rather than the tracking script loader.

i opened the article and "oh, really?"

I'm amused by the fact he thinks `array.push(arguments)` inside the nested function evaluates and pushes the arguments from the parent function.

Yeah, I'm surprised noone had pointed this out.

For all those here mentioning that he's only "deconstructed" the snippet, and not the actual main linked analytics script, you're the first to observe that he's deconstructed the snippet incorrectly.

Don't get me wrong, it's really great and heartening to see someone new to JS getting back to basics and trying to understand things in proper minute detail (instead of doing another tutorial about a boilerplate React app), but this is hardly HN front-page material? It's a very simple JS snippet.

Tried to search but nothing obvious showed up. What's the purpose of the 1 in '1 * new Date()'?

Edit: found it. Gets the timestamp[1]

1. https://stackoverflow.com/questions/24182317/multiplication-...

Right, back to the point of saving characters/bytes at scale it's a cheaper way to getTime()

And if you don't need to support IE8 or lower, you can use Date.now()

Some do analytics tools use +new Date instead

On a related note, there is an open source Google Analytics alternative in the works:

https://github.com/vesparny/fair-analytics

Hope it comes along; there is surprisingly not much else in the way of Node/JS based open source web stats/analytics solutions.

Piwik is an open source GA alternative [0]. It doesn't share many of the goals of Fair Analytics. I also don't see much actual analytics in Fair Analytics, just data collection.

https://piwik.org

Yes, and it's available on https://sandstorm.io so you don't have to host it yourself either.

I actually much prefer the Piwik interface, I don't need to dig through masses of menu options to find the data I actually care about. Not helping the Google panopticon is a bonus.

The thing with piwik is that it's

1. Has a large footprint (won't run on tiny instances) 2. Relies on blockable user side JS.

Is there something like that that just analyzes server logs?

Back when we walked 5 miles uphill both ways to get to school, server log analysis was all there was. I think AWStats [0] was the most prominent.

But Piwik can use methods other than JavaScript in the browser, including importing server-side logs [1].

[0] https://awstats.sourceforge.io

[1] https://piwik.org/faq/new-to-piwik/#faq_63

Agreed, FA looks like raw material compared to piwik. There's a lot of components needed to build a full solution. You need browser capture, data sink, backend aggregation, dashboards, adhoc query capability, etc...

Snowplow Analytics - https://github.com/snowplow/snowplow

+1 for snowplow. really cool system

I really don't see that being used on anything but hobby sites. Their definition of privacy essentially means it doesn't work.

For me, Google Analytics privacy issue is that it's used everywhere - Google can track users across a fair chunk of the web. Your site running its own private Piwik install? Fill yer boots, not a problem.

I have minimised it in my own case to just this (using ga.js instead of analytics.js because this part is shorter—regardless of the fact that ga.js itself is longer—and not worrying about any fanciness that I don’t need):

  <script>_gaq=[['_setAccount','UA-????????-?'],['_trackPageview']]</script><script async src=//ssl.google-analytics.com/ga.js></script>

With analytics.js:

  <script>ga={q:[['create','UA-????????-?','auto'],['send','pageview']],l:+new Date}</script><script async src=//www.google-analytics.com/analytics.js></script>

The reason it is a function that calls itself is to not pollute the global namespace. The rewritten expanded example leaks "gaScript" to the global namespace which is a little rude if you are injecting your script onto a third-party page and using a random or long function name is not as clean a solution since it still leaves dirt and takes more bytes.

Another interesting aspect of Google Analytics is understanding how fraudsters sneak by, and the discussions here on HN whenever this comes up.

Examples:

· Hackers Make $5M a Day by Faking 300M Video Views | https://news.ycombinator.com/item?id=13219871 (6 months ago)

· Uncovering an advertising fraud scheme | https://news.ycombinator.com/item?id=2333824 (6 years ago)

Discussions:

· Alleged $7.5B fraud in online advertising | https://news.ycombinator.com/item?id=9796102 (2 years ago)

· Inside Google's Secret War Against Ad Fraud | https://news.ycombinator.com/item?id=9628967 (2 years ago)

Here's something fun for people who wan't to customize what it says instead of 'isogram' (a fun little easter egg)

https://isogrammer.com/

> Interestingly, the arguments passed to the function spell out i, s, o, g, r, a, m, is a “term for a word or phrase without a repeating letter” (source), which I guess makes sense, given that the script looks like it has the bare minimum of characters possible.

I thought that was a kind of recursive joke (not necessarily a very witty one) : if they want to make the parameters spell out a word, the word has to be an isogram, otherwise you'd define a parameter twice. And since "isogram" is an isogram, maybe the temptation was too great..

Looks like the insertBefore call got dropped along the way.

> It seems like a and m are optional arguments. > Now those unused parameters a and m come in handy. No writing var in this script.

Precisely. A trick to reduce the character count, nothing more. Love doing this kind of thing (http://gabrielgambetta.com/tiny_raytracer.html).

> Making a.async truthy ensures

"Truthy"???

Truthy on MDN: https://developer.mozilla.org/en-US/docs/Glossary/Truthy

Truthy means that if(<truthy value>) evaluates to `true`, but that the value is not literally `true`.

In JavaScript, "falsy" is null, undefined, 0, '', or false. Everything else is truthy, including empty objects, strings with only whitespace.

Many bugs have been caused by incorrect uses of truthy and falsy.

Also `NaN`.

A tool that generates tracking codes with a customised isogram as parameters: https://github.com/shinnn/isogram

Example: (function(i, s, o, g, r, a, m) {...}) -> (function(y, c, o, m, b, i) {...})

I can't think of anything less useful/productive/meaningful.

this comment, for a start.

I'm not sure if it's actually intentional, but given the command-queue nature of the ga function, i[r].q's resemblance to IRQ could be a cute reference. (Assuming i and r were fixed, p=1/26 for using q... science!)

(Originally noted in some old article that I can't find right now.)

I also liked this deconstruction: http://code.stephenmorley.org/javascript/understanding-the-g...

> An unminified and less convoluted version of the script might look like this:

(references to undefined variables `a` and `m` in some sort of IIIFE)

Sherlock here discovered that it asynchronously loads the actual script, which he conveniently did not bother to explore further.

Yes, and when I was 5, I took apart my parents rotary telephone and figured out how it worked. I'm sure there was a document somewhere that already explained it... but for me, actually "discovering" it led to a new level of understanding. I'm confused by your use of "Sherlock" as a pejorative. I actually like the author's use of deduction as a learning tool... instead of just vomiting knowledge read elsewhere.

> I'm confused by your use of "Sherlock" as a pejorative.

It's not a pejorative, it's sarcasm. You could replace "Sherlock" with "This genius" in that sentence for essentially same effect.

I'm not really sure why, but for some reason "Sherlock" has become very common to use in a sarcastic context like that. Kinda like how "fat chance" and "slim chance" now mean basically the same thing due to "fat chance" almost always being used sarcastically.

Regardless... even though what the author discovered is common knowledge to those who've done some work with GA tracking (or perhaps simply researched interesting ways to add elements to the DOM). I applaud the curiosity that lead to this blog post.

Nah. The author did the equivalent of saying they would tell us how the iPhone works, but got as far as just opening the box it comes in and powering it on.

It is pejorative sarcasm

> instead of just vomiting knowledge read elsewhere

that's a nice line and describes some very high karma HN denizens.

perhaps, but the notes about using unused args to avoid var declarations was a neat hack.

for more on async loading techniques, this is a great blog post.

http://blog.errorception.com/2011/08/reliable-high-performan...

Reply on news.ycombinator.com