GoogleBot runs only in a few datacenters in certain locations in the world with amazing connectivity.
It is a horrible predictor of load times.
It is a horrible predictor of load times.
Seriously?
Not every city; the top 500 cities would probably cover 90+% of the users.
You're not understanding the scale of Google. They're incredibly good at exactly this stuff.
Is a cable is cut between Jakarta and Singapore, it can take months to fix as Indonesia only allows Indonesian-flagged cable layers to operate in its waters.
Individual datacenters, CDNs, cables are going offline constantly, so you'd need to be continuously testing every site to rank it correctly. Not only that, each page in a site might have a very different structure, so you can't just rank a site, you need to rank individual pages and articles too, which means testing every new page and article from every city.
I guess you're not getting the scale of that...
AMP standardizes the CDN serving part, largely eliminating that issue. AMP articles can't prevent 3rd party CDNs from caching them, which means that they can be served from the closest CDN after the first time they are used.
Which is the reason why Google focuses so much on the number of roundtrips. For most, downloading 50kbyte doesn't cause a lag but 10 roundtrips before the site can be rendered do.
An user in Indonesia isn't at the closest datacenter in Singapore. It doesn't matter what you do, you can't, from Singapore, try to emulate what an user on a particular ISP in a particular city in Indonesia will face.
In case it isn't clear:
Consider 2 websites, both have hosting in Singapore, but one also uses a CDN in Jakarta.
The Google crawler running on the Singapore datacenter connects to both websites, speeds are great because they are sitting just next to the datacenter. Both get great ratings.
An user is in Jakarta, and tries to connect to both websites. One of them takes a century to load, because it has to load everything from Singapore, and international cables from Singapore to Jakarta are expensive, so connections are capped. Experience sucks.
The second website uses a CDN in Jakarta, located at the local coloc facility to which the user's ISP is also connected to. The website loads instantly.
CDNs today are located in every single small ISP, in all countries, in all medium-sized cities in the world.
If the problem is a lack of information about global network conditions, then you can deploy canary machines around the world to gather that information.
> If the problem is a lack of information about global network conditions, then you can deploy canary machines around the world to gather that information.
You'd need to put machines in every single city and every single ISP in the world continuously testing every website in the world.
Doesn't make any sense at all. That's not how networks work.
If you're in Jakarta trying to access a certain website, if that website is available at the local CDN might make a bigger difference than which ISP or even which plan you're using.
You could have a 1Gbps fiber connection from your ISP in Jakarta. If you're trying to access a website in Singapore the your ISP's international link is congested, that 1Gbps will be useless.
OTOH, if you have a 10Mbps connection, and you're trying to access a website replicated at an Akamai CDN in your ISP's network, it will be way faster.
Correction: Google can't predict the speed of a random webpage to a random user around the world.
But you can reasonably predict that pages hosted at widely available CDNs around the world will load faster on average than pages not available.
CDNs are a hack that can make websites load a bit faster, but that's about it. It won't make them use less RAM, or consume less of your data cap. Ultimately, performance is about the size of the site, the resources it uses on the client, and the number of requests it makes. All of which is independent of the number and location of CDNs.
It does, because the definition of bloatness depends on the network speed.
What counts as bloat for an user sitting in Singapore isn't the same as an user sitting in Jakarta.
> Beyond the fact that the network can be mapped (Google surely does this) and simulated
Network topology is irrelevant in this case. The ISP in Jakarta might have a satellite connection to Singapore, but have a CDN and it will load faster than in another ISP with a 10G international fiber link to Singapore.
> All of which is independent of the number and location of CDNs.
That's completely wrong. Today, the vast majority of internet traffic is served from CDNs, CDNs (and their placements) are critical for the internet as we know it.
No, it doesn't. It depends on the proportion of actual content to the unnecessary cruft. The difference is that what a Singaporean considers bloat might prevent someone in Jakarta from accessing the site at all.
> That's completely wrong. Today, the vast majority of internet traffic is served from CDNs, CDNs (and their placements) are critical for the internet as we know it.
That is if you include streaming video, which is off-topic for this discussion.
Imagine a perfect world with the ultimate CDN solution - something like IPFS, which ensures everyone is a CDN to everyone else (instead of wasting bandwidth with multiple downloads of the same content). The concept of bloat wouldn't disappear in such a world, because bloated websites will still require you to download unnecessary amount of data to your machine, and will reduce your computer's ability to multiprocess while also using up your battery life.
I stand by what I said before - CDNs are a hack (basically a hand-rolled, rudimentary distributed system) to give you some extra download speed. They don't solve the problems of download size and resource usage.
If you pretend that all your users have net connections like the US, Europe, or parts of east Asia then you're going to end up as a small player in a balkanized internet. That's why all these companies are making aggressive investments and acquisitions in other parts of the world.
CDNs are also part of the fundamental reality of the internet. Could you imagine Netflix or YouTube without CDNs? The amount of network capacity investment necessary to run the internet as we know it without CDNs would cripple it.
And it's not just the internet, but it's a fundamental fact of computing that you have hierarchical caching layers with some data far away and some data closer.
It's not about privilege or capturing emerging market. It's about laziness and waste, pure and simple. The same bloat that prevents someone from accessing your site on a smartphone in the middle of Africa is also significantly slowing down the computer of someone in western Europe, and eats into battery life too.
And let's not bring streaming video into this as it's completely off-topic.
And that's the privileged position. Bloatiness is annoying if you're an American reading American pages, or a Spaniard reading Spanish pages, and we like to complain about it because it means the difference between being able to read an article on the subway and having to wait for the next stop. But I'm never in a position where I have to complain about the backhaul network or the backbone.
In the future, please don't try to police "off-topic" conversation.
Public web sites that only work for "first world internet" customers shouldn't be a thing. Sites that load 100MB of ads and trackers and fifty different JS frameworks to display a few paragraphs of text should be getting the search-engine death penalty until they clean up their act. They should be thinking about people who aren't on fast broadband connections. If they aren't, the search engines should be.
YouTube learned this from the "paradoxical" result of a lighter-weight page increasing load times:
http://blog.chriszacharias.com/page-weight-matters
(spoiler: it was because the lighter-weight page meant it was suddenly usable by people who had slow connections who couldn't use it previously, and those people began to use it, which skewed the average load time higher)
Let's say your website is hosted in Jakarta, and serves Indonesian customers. You make it lightweight and fast.
Your competitor hosts its website from Singapore, and is much heavier.
Now Google, from its datacenter in Singapore, wants to test the load times for your website and your competitor's. Sadly, the host you chose doesn't pay for a lot of international traffic on the cable to Singapore, since it primarily serves the Indonesian market. Google tests both sites, and your competitor's, based in Singapore about a mile away from the Google datacenter, loads several times faster, besides being heavier.
Google ranks your competitor higher, and you are pissed off, as your customers load your site faster.
Now another case: you and your competitor host your sites in Singapore. You build a lightweight site, and your competitor's is full of heavy assets, with oversized images, etc. Google tests both of your websites and yours loads faster, and is ranked higher. You're happy.
But your competitor signs up for Akamai and replicates the heavy assets in local servers in all the major ISPs in Indonesia.
Customers in Indonesia search and see both your and your competitor's results. You rank higher and get more clicks, but as your traffic is subject to the congestion on the international cables, your site actually loads far slower than your lower-ranked and heavier competitor. Customers are unhappy.
In all of this, simulating a slow connection on Chrome's dev tools is irrelevant, because you don't know where the bottleneck is!
Simply simulating a globally slowed-down connection will tell you what's going to be a problem. And, really, common sense will tell you what's going to be a problem, too -- if a site "needs" to load 100MB of ads and trackers and other JavaScript to display a couple kilobytes of actual text content, then you have a problem. A user on a slower connection is going to see that site take forever to load even if there's a CDN across the street.