514 karma · joined October 2, 2019
The "shard" could just be that users own feed in this case. Then you get offline for free where user adds a tweet and it appears immediately, replicating back to server when he goes back online. The server replica side will need to be a lot more complicated to deal with broadcasting but I don't see why it won't work.
Instead we have a bajillion layers of CRUD all in slightly different protocols just to do the same read or write to the database.
I tried to find an example of this and found a somewhat related article about how Japanese grammar maps nicely onto ruby: https://thoughtbot.com/blog/learning-japanese-the-rubyist-wa...
I am not so sure about that. Have you seen this thread: https://www.reddit.com/r/MachineLearning/comments/dfky70/dis...
Apparently lots of sentence fragments were memorized in GPT-2 (including real world URLs, entire conversations, username/emails and other PII).
For example their “Colossal Clean Crawled Corpus” (C4), a dataset consisting of hundreds of gigabytes of clean English text scraped from the web, might contain much of the same information as the benchmark datasets, which I presume is also scraped from the web.
I am just trying to support the grandparent comment that the current optimization is at the wrong level of abstraction. React is at the level of a "scene graph" (dom). If it's slow, the first technique to reach for should be stuff like "occlusion culling".
As is tradition, frontend developers are still slowly reinventing the techniques from other fields.
The neat thing about UI is that your end user is a human and a human can't visually process too many items at once on a screen. This means you should never really need to consider more than O(thousands) elements to render either. That should always fit under 16ms.
This has been known to game developers since forever. Don't spend resources on things that won't be displayed. If you are doing too much work that means you aren't culling invisible objects or approximating things at a lower level of detail.
In practice for frontend developers, this just means using stuff like react-virtualized so that even if you have a list of a bajillion objects, only the stuff in the viewport is rendered.
But with mathematics, "intuitive" analogies are all in terms of other mathematical objects! You can't build intuition if you don't even know what they trying to abstract over.
In that regards, The Princeton Companion to Mathematics is fantastic because it maps out how the different fields of mathematics are interrelated.
I wonder if React would've needed to go down this path if they were built by a different company.
For example Google/Apple/Microsoft all have huge influence over browser implementations (chrome/safari/edge respectively). They might've been able to push a ECMAScript proposal for proper language support instead.
It's a very clever hack regardless but it only seems necessary because Facebook doesn't have control over the whole stack.
(it does seems like Facebook trying to fix this by building their own React-optimized javascript engine but only for react native on android right now: https://facebook.github.io/react-native/blog/2019/07/17/herm...)
That clock animation is pure genius:
For example in "The Problem of Induction" you can see a bajillion sunrises but can't infer that there will be another. Likewise you can conduct a bajillion controlled experiments and can't really tell if the result will generalize. Worse yet is that the more conditions you control, the less likely it's to generalize. Infamously, clinical trials wanted to control for the variations caused by periods so they only tested on men and it turns out some drugs affect women differently! In general, if you control for everything, you can't say anything about the real world outside of those precise lab controlled conditions.
But despite every big company throwing a couple billions at the problem for the last few years (Facebook's $2 billion acquisition of oculus, Google's $500 million investment into Magic Leap, Apple's acquisition of Metaio/ARkit, Microsoft Hololens), all we have are literal toys!
So now you can't talk about AR as a serious subject anymore. People all dismiss it, saying we are still too early. I imagine it's like that for every "toy to big thing" transition.
I assume you didn't mean that literally because I can't see how that will ever work out in terms of cpu cost. I think breaking it up into blocks like what RAID4/5/6 would be better but will still impact the performance of reads.
The performance of writes is going to be worse. Not because of the parity calculation but because you will be taking the max latency over all the cloud providers.
I can't see people trading off that much performance for better fault tolerance (in a world where S3 guarantees 11 nines) or ease of switching.
> Chinese investors bought about 70% of free-standing houses on the west side of Vancouver in a six-month period
And 40% of the population is chinese:
> 40% of the residents of a large portion of Southeast Vancouver are Chinese.
https://en.wikipedia.org/wiki/Chinese_Canadians_in_Greater_V...
Cloudflare doesn't charge for bandwidth. I always throw cloudflare on top of anything I do, not because I really need a CDN or anything, but because the bandwidth cost would bankrupt me otherwise. The ceo of cloudflare gave the rationale on why they don't charge:
> There’s a fixed cost of setting up those peering arrangements, but, once in place, there’s no incremental cost. That’s why we have similar agreements to Backblaze in place with Google, Microsoft, IBM, Digital Ocean, etc. It’s pretty shameful, actually, that AWS has so far refused. When using Cloudflare, they don’t pay for the bandwidth, and we don’t pay for the Bandwidth, so why are customers paying for the bandwidth. Amazon pretends to be customer-focused. This is a clear example where they’re not.
I am looking at MIT's salary survey and they have similar ratios: https://capd.mit.edu/sites/default/files/about/files/GSS2017...
Out of 170 graduates, 27 went to facebook, 21 went to google, 13 went to microsoft, 6 went to amazon, 2 went to apple, etc. That's basically half the class including a lot of people from the bottom half of the class (the best people I knew usually didn't choose FAAMG and their outcomes were a crapshoot).
They are usually hugely influential in their times but have been refined or made easier to digest since. If you care deeply about your field you should read them at some point to see how ideas evolved historically.
English: Shakespeare
Biology: Darwin's Origin of Species
Math: Euclid's Elements
For OP's topic, I'd say Shannon's "A Mathematical Theory of Communication" is up there for computer science.
So it's important to understand why humans believe in metrics. Humans by default are multi-objective. We don't care about just one or even a few things, we care about a lot of things, all at once, with nobody agreeing on what they are. Identifying the "true direction" we want to go as a collective is impossible. So we pick a good enough direction so everyone can be somewhat aligned. To make it possible to communicate this vision, we compress the world down to few dimensions even though we know that it is absurd. This works fine for humans because our "distributed system" is composed of beings with enough intelligence to notice/alert/convince the rest of the system whenever we are veering off course.
Whenever we program AI with these metrics, we forget that we can't direct them we way we direct our sentient brethrens. Any simple metrics are necessarily doomed to fail because it doesn't have this self aligning property. So ultimately it is the programmer's responsibly to go and realign the system back with human values. If necessary, with some social pressure (hence this article). It doesn't seem like there will be any other way.
With node.js: 800 containers, with each container processing 250 messages per second
With golang: 340 containers, with each container processing 650 messages per second
Say each one of those containers cost $0.02/hr then that's order of $100k/year saved!