Open sourcing anything at Google tends to be rather difficult because much of the technology we build is built on other technology that has not been released. It tends to take a great amount of effort to open source anything at Google because of all the internal dependencies you have.
It should be noted that Prometheus is an open source recreation of the precursor to monarch (developed by a former Google SRE that has since returned).
Monarch might be an exception to that, though, because - as noted in the paper -, it has to be built without many dependencies, since almost everything else depends on it for monitoring and alerting.
That being said, I'm sure it would be extremely difficult to open source now since it was never built with open sourcing in mind.
In the case of PageRank, at least it was published after Google had already dominated the search space, i.e., many years after the conception, implementation, and utilization.
I wonder if Google papers are internally reviewed before publishing so as to make sure only partial information is revealed, and not the secret sauce.
They aren’t worried about ‘giving away the idea’, they just don’t have an easy technical way to open source just the one component.
The thing is Monarch is not the secret sauce that makes Google money. Every other company in the world could be running monarch, and Google revenue and market position would not be affected.
Publicizing this is a win-win situation. The idea gets exposure and potentially other devs can be benefit, and Google gets some good PR as a place where devs get to work on cool stuff.
The issue is, even if you give away the secret sauce that doesn't really help with making the secret sauce scale or whatnot, nor does anyone that isn't a large cloud provider need a custom solution like monarch. Prometheus or Datadog work fine for everyone else. This might be interesting reading for those companies, but also it might not be, because those can't be as centralized as monarch is (consider if prometheus had an API and ran a centralized cluster of data-ingestion servers, and you made time series to that global, Prometheus-owned, cluster).
I'm not actually sure that's true. It's like many other things inside Google - people outside don't necessarily understand the value or know what's actually possible, because they've never experienced anything similar. It's sort of like trying to discuss the finer points of the taste of oysters with someone who has never tasted them.
I would very much like the feature set of Monarch (and streamz), without the maintenance overhead or even insane scale. Very, very few companies out there need to run anything at anywhere near "billion-user" scale, but literally all of them could benefit from painless and detailed monitoring that Monarch offers.
To the best of my knowledge, nothing else in the market does that.
https://cloud.google.com/monitoring/api/ref_v3/rest/v3/Typed...
Since the distribution is represented by a CDF of buckets, there's no guarantee that you'll get an accurate representation of the median or any other quantile. On the other hand you'll get an exact average.
Pet peeve: it can calculate and graph something it calls a quantile. But if the value is in the middle of a large bucket, it will just interpolate or something and the result will be terribly misleading. It'd be much better if it gave lower/upper bounds.
I try to use distributions via questions of the form "what fraction of values are less than / greater than N [which I've verified is a bucket boundary]?". This gives you an answer you can trust.
The core idea is a distributed time-series DBMS... not much to give away there. It "gives away" some architectural novelty, but it's not the solution to P=NP. These papers typically describe engineering feats more than they do a revoluntionary idea.
As impressive a the Planet-Scale sounds like, monitoring system has been essential in a lot of companies, big or small, and there are tons of companies dedicated to this problem domain.
So it is not an untapped territory for many. Google solved a problem that is relevant to Google and its scale. The solution is cool, but it is only so because the problem exists in the first place.
Without Google's problem, the solution will be seen as incredibly over-engineered and unnecessary.