samhw

Yeah, I appreciate what 'auto' and 'scaling' signifies. I've implemented an autoscaler on a huge Kubernetes cluster in the past. That's precisely where my doubt comes from.

I was about to write out a huge example, but I figure I'll just express the core logic simply. First, it takes a chunk of time to determine that increased traffic is not just random variance. Then it takes time to allocate and provision machines. And often the traffic spikes for you and for your co-tenants are not statistically independent, so the provider struggles to allocate machines in time when it most matters.

And how much do you scale up? 10x right away? Can't do that: vastly expensive, and anyway it could be a retry storm exacerbating things. 2x and then go from there? Well, if your business is serving ads during the Superbowl break, that's not gonna work. Etc. This all starts to look increasingly absurd against the backdrop of being able to just push a button and do it yourself.

I'm not suggesting never doing autoscaling. I'm just saying that wise men don't speak in absolutes, or make architectural decisions based on toy examples. Nor am I particularly bothered about whether I'm doing things "like it's 1996" - I'm not in the fashion business, so I'm purely interested in finding the optimal solution to my problem, and I couldn't care less whether it's flaming hot or whether it's from the Iron Age.

Yeah, I'd agree with that. But even then, it would be more valuable if it committed to at least some meaningful, non-platitudinous positions. Something with which at least one person on the planet might disagree. Not "it would be nice if databases were fast, scalable, reliable, personally tailored to everyone on the planet, ...".

For example, my predictions for future databases would be:

- They will take over more functions of the average backend codebase: instead of database users, they will have a concept of application users, along with their privileges, and simple CRUD logic will be executed by the database.

- Horizontal scaling will be less important than we currently think. Consensus will be handled at a lower level, by networked filesystems or storage engines. (Zookeeper, in the Java world, is a proto-example of what I mean.) This is one instance of the trend that...

- Databases will be 'unbundled' (Kleppmann's term). Many of the dull uniform bits will be shared rather than reimplemented. This will happen either through libraries or - more likely and preferably - through separate pieces of software, implementing an interface, which the user will compose. (Rebundling will occur for users who just want a click-and-tick experience.)

- Self-optimising will - I agree with the article here - widen in scope. Users won't have to perform housekeeping tasks like creating indices on commonly-queried fields.

- Databases won't target a filesystem but block storage. This will accompany a convergence of disk (NVMe) and RAM (NVRAM) towards persistent random-access storage of state. Databases, along with applications, won't think in terms of a rigid distinction between "what's in my process's memory" and "what do I have to expressly commit to the disk with a syscall". Tuple spaces are a precursor.

> maybe you need optimization query but not to deploy databases, migration and that

Well, how do you indicate to it that you want to create a database? Does the database read your mind too? How do you indicate that you want to provision some more because there's a big event coming up?

> Scale should not come at the cost of performance is easier said than done

That's essentially my exact point, yeah. It's not a novel aspiration, and they don't contribute any kind of solution. It's like saying "computers should cost less and also be gooder".

> i think the same i will still use monolith like PostgreSQL

Yeah, I think that's probably sensible for many people, though I'd also welcome more innovation. Kleppmann's idea of 'unbundling the database' – i.e. the modern database becoming fragmented into several distinct components – is I think very promising and very probable. (Of course, your business and your production environment may not be somewhere you wish to be a hotbed of experimentation.)

I can't make sense of lots of this.

- Without self-replicating grey goo, infinite scalability is surely more a property of some kind of networked computer rental business (like AWS) rather than a database.

- What does 'serverless' mean exactly? My understanding is that it denotes a stateless application which is executed to serve a request but doesn't run continually as a daemon. Essentially the aforementioned computer rental business provides the event loop and the program provides the event handler. I fail to see how this is compatible with a database, which is definitionally very stateful. (And that encompasses much, much more than just the data.)

- 'Intelligence': Databases already are intelligent and self-optimising, and have been at least since MySQL/Postgres: https://dev.mysql.com/doc/refman/8.0/en/cost-model.html#cost...

- 'Fundamentally reliable': This idea could reasonably be described as, uh, 'not novel'.

- 'Distributed globally, locally available': As far as I can tell, this collection of words is entirely devoid of any meaning. It sounds like it came out of a random passphrase generator.

- 'Scale should not come at the cost of performance'. While technically semantically meaningful, this is not novel or interesting, and I'm pretty sure this has been a pleasant daydream for database designers since databases were stored in punchcards. As far as I can see, this is comparable to saying 'houses should not come at the cost of money'.

This feels more like a laundry list of daydreams rather than a meaningful narrowing-down of how future databases will be architected. "It should be infinitely scalable, usable by a toaster, and it shouldn't need a computer to live on. It should be serverless and stateless but also self-optimising and with connection pooling. It should be consistent, available, and, uh, partitions, it should be cool with those too. It should be usable by anyone and perfectly tailored to their needs as well as to the opposite needs. Also..."

How on earth is TikTok a competitor to Netflix?!

I dunno. I'm not so unhinged about it as the guy above, but I do share some of his exhaustion at the ideological box-ticking of these TV shows.

I'm left wing and I believe everyone deserves equal rights and equal respect, but I find utterly fatuous the idea that 'representation in television shows' is an important political concern. I believe in fighting global starvation, climate change, social safety nets under increasing automation, etc - not how many transgender lesbians are in TV shows.

I do not give a fuck about that, I do not give a fuck if transgenders and/or lesbians give a fuck about that, it is not important, and I'm disturbed at the increasing frivolity, corporate-ness, and lack of depth in political life when people are seriously suggesting that this is a worthy expenditure of our political willpower and attention (which is extremely finite and scarce, regardless of what people on Twitter insist).

> fear it ending suddenly without resolution

I don't mean to sound like some "well akshuallyyy"ing Comic Book Guy figure, but you must realise it's not real life? If it's a Poirot-type mystery where the entire payoff lies in the resolution, then I totally understand, but for a regular TV show I can't see why you would enjoy the episodes any less simply because it might end without resolving all your questions. There's no real suspense - it's not real people, it's just a bunch of people performing in front of a camera. Being distressed about 'what happened to them?' seems, well, puzzling to me.

I don't know why this would even be paid from company money. If I Epsteined my flight-attendant-cum-masseuse, my company would not pay for that. I don't think I would want my company to pay for it, given the privacy/leaking implications. This seems deeply odd to me, even as someone already vaguely sceptical - from what I've read - of Musk's financial situation.

If you change programming languages based on random comments from loonies on the internet, I don't think you're ever going to stick with a language long enough to implement hello world...

As far as I know, we do lifetime licenses in the UK, aside from photo licenses where the photo has to be renewed every 10 years (which can be done online). However, our driving tests are quite different from those in the US, in that they actually test your driving. (When my aunt and uncle moved from London to LA, they did their driving tests while severely jetlagged and still passed...)

Also, eight seconds sounds like a terrifically long time for it to take effect. That's tens of billions of CPU cycles, billions of arithmetic instructions, millions of disk seeks, a hundred or so page loads, a dozen "knock knock" jokes in call-and-response between London and Sydney. How non-technical does one have to be to think "eight seconds" is fast in computer time?!

> I guess what you're saying is that apprehension about the motives of of corporate executives is so irrational, it's akin to a psychological problem.

No, that wasn't what I was saying there, but coincidentally, yes, I would agree with that statement. For one, 'corporate' is a phenomenally lazy word used to invoke some mental image of besuited sociopathy, from a million children's movies and internet debates. In the real world, 'corporate executives' - human beings, mothers and fathers, working in management careers at (large) companies - are humans doing jobs, like any other humans doing jobs.

I've been involved (as an engineer) in C-level meetings with people like that, as well as worked alongside them in the same teams, and my experience - thoroughly unsurprisingly given what I said above - is that they are pretty ordinary human beings on the whole. I'm sure there are differences between companies in terms of their cultures or values, but that's company-specific and not discipline-specific or seniority-specific.

I get that your experience is that these are people who mete out decisions from on high, which affect you in sometimes-negative and always-unforeseen ways, and that makes you feel like a mediaeval peasant might have felt towards his feudal liege. And maybe it's helpful, from a cathartic standpoint, like I said above, to picture them as evil baddies from kids' movies. But no, the reality is more banal.

(Incidentally, I think those who worship company CEOs - and suchlike - are equally stupid, from the Elon bros to the HackerNews types who recite stories about startup founders as though they were superheroes. These people underestimate the role of chance and – in a way, very similarly to you – overattribute the differences between execs/founders vs regular workers to a proportionate difference in their inner essence [in their case, they think intelligence & industry; in your case, you think amorality]. Both are fantasies.)

I think that definition is right, but only where 'simplest' factors in the probability of each premiss, not just the aggregate number of premisses. And yeah, on that reading it's trivially true, but something's being trivially true doesn't mean humans as psychological beings don't need reminding of it.

Consider what we see if there isn't enough information: https://en.wikipedia.org/wiki/The_Market_for_Lemons And on a totally separate note, consider what we see in DeFi.

I can't see why Occam's razor wouldn't apply: don't postulate more entities than are necessary to explain the facts. Specifically, there is no reason to postulate some kind of anti-Firefox elements within Cloudflare when it's simply explainable as a mistake.

(There's a reason Hanlon's razor was given that name, I'm pretty sure, and that's because it's not too dissimilar from – essentially it's a much-more-situation-specific subset or variant of – Occam's razor. But I don't think "attribute to stupidity" was what the parent comment was suggesting.)

Honestly, do you want a BigCo job? Would it make you happy? The experience of being a tiny cog in an unthinkingly vast machine is more alienating than you might think, even if you like the money.

I think lots of those who stay on long-term in that environment, say >3-4 years, effectively become institutionalised, condemned to spend the rest of their life philosophising about whether they can ever be L8 material. These are people who are, by and large, smart but not wise. Yes, the money is good, for an employee, but it barely equals what even a small business owner could make - and these are people whose smarts would allow them to thrive in the software niche, at least with a minimal amount of commercial nous.

The truth is that FAANG (or FAAG, considering what's going on at Netflix?) keeps these people there by creating a value system. If you've ever read the internet-classic 'Five Geek Social Fallacies' post (https://plausiblydeniable.com/five-geek-social-fallacies/), you'll know that people like us - frankly - are suckers for taxonomy. If you tell them that they are the best because they work for you, and the only thing that can make them any better as an engineer is creating maximum returns for the Borg, and surround them with others who mirror and validate that behaviour, they will fall for it for the rest of their lives.

Don't fall for it. There are a fuck ton of smart people at FAANG companies – that's one of the few marketing points which is unvitiated in its merit, they do scoop up very many of the smartest (again, smart-not-wise) people in the world – but that doesn't mean the only way to be smart is to join. It doesn't mean you - as a unique human being - will be happy there. Carve your own path, be suspicious of people selling you axiologies and hierarchies, find a middle road between excessive cynicism and excessive credulity.

Yeah, it's clear these people have never hired for a company. I've sat in on a thousand hiring committees, helped design these processes, all that boring shit, and the idea that someone would say "let's do Leetcode challenges so that we select for people who are willing to subordinate their lives to our company" – presumably followed by a "mwahahaha!" – is fantasy, it's something out of a movie.

I mean, look, it sucks when you can't get a job. I've been there (when I briefly tried to escape programming for a different metier) and it's awful. But pretending that the entire outside world is a dark mass of malevolent forces arrayed against you.. well, it may be psychically necessary in some Freudian, Jungian sentence, to prevent ego death or whatever, but in the long run it's neither (a) true (obviously), and (b) nor is it going to do your mind and mindset any good.

> Maybe what Rust needs is a “build a program with lots of allocations” and then “whittle down allocations with smart use of references” section of the book.

Rem acu tetigisti. This is what needs conveying. I think with Rust people are often speaking past one another. One person says "hey, you can't write EVERYTHING with meticulously-planned-out lifetimes, cows for every string, etc", and what they have in mind is a junior/novice programmer's first-time experience (or any combination thereof).

Whereas another person says "you can't clone everything everywhere, or wrap everything in a RefCell", and they are right about what they mean: that properly written software eventually, to be respectably efficient, needs to replace that kind of code with code that thoughtfully uses appropriate lifetimes to truly benefit from what Rust provides.

As usual, Wittgenstein is right, specifically in what he says in s.4 of his famous Buzzfeed article[0]: most so-called problems are merely confusions of language, where two people mistake an inconsistency in their use of signs for an inconsistency in their opinions. If they set it all out more fully and explicitly, I don't think there would be much disagreement at all about what's appropriate, at least among the 80% of reasonable people.

[0] https://www.buzzfeed.com/ludwigwittgenstein/fantastic-ways-t...

>> This is what generics are for in most languages. > That goes exactly opposite of this optimization, and is just entirely orthogonal.

What does that mean? Also, how is it "exactly opposite" and "orthogonal"? Maybe let's get rid of the mixed metaphors and speak in concrete terms.

In your comment below ("why would you want the compiler to do that? you can write out all the monomorphizations yourself!") I'm not totally certain that you understand the point of generics. You can always write out each case yourself. Incidentally that also applies to your precious inlining. But it's a tool to make programmers more productive.

This was hilarious. I hope it was meant to be. I liked the parody of the deluded man who thinks he's the best and that no one recognises his genius - maybe it was a bit on the nose, a bit too cliché, but nevertheless. It was a nice touch that the character spends, like, a thousand words saying nothing at all besides "I am smart, Netflix hires smart people, like me, I am smart, Netflix doesn't hire me, but I am smart enough that I will eventually make this syllogism make sense, I am smart, also I am smart".

It also explains quite well why I don't hire people like your character. They think they aren't hired because of their arrogance, like it's a downside that only merely vitiates their tremendous intelligence. In reality an attitude like that is a nearly-perfectly-reliable signal of someone who's totally deluded about their own intelligence. Truly super-intelligent people of course are not chippy, insecure, oppositional, constantly expatiating upon their great intelligence, etc - it's just a given, a neutral fact, much like the way Bill Gates doesn't go on about how much money he has, whereas a recruitment consultant earning 50k a year will go on about such.

Anyway, I enjoyed the skit. Thank you for sharing it. The technique of your not using quotation marks and suchlike really imbues it with a sense of great immediacy and realism. Marvelous stuff.

Yeah, this is the obverse of why I don't go to the US. I know the guy upthread was smarmily pushing the "oh, it's just because you CAN'T get a visa / get a job / figure out how the fuck you're meant to apply for an ESTA / etc" line, but it's not really that. I'm at the not-literally-top-percentile-but-very-very-close mark in London, and I'm confident I'd be paid commensurately in the US. But the healthcare situation is an absolute, hard no.

I don't care if I get private insurance in the States. I get private insurance here too, and I even use it now and again. But I care about knowing that, no matter what happens to me, ever, I won't be saddled with a life-ruining debt that wipes out what I've gained. Not without my consenting to borrow such an amount.

I'm not interested in playing snakes and ladders. I'm not interested in the equivalent of someone selling me a mortgage without my knowledge in my sleep (and with no house at the end of it). And I know that insurers - unlike the UK govt - are in the business of avoiding making payouts[0].

(That's the main thing, at least, by far. Then the salary-erasing real estate costs, in all the places where those endlessly-quoted salary figures are from, make me not-particularly-sad that I ruled it out.)

[0] I'm not imputing dishonesty. I'm just saying their literal business model is maximise premiums, minimise claims.

Bugs? Roll back. Bugs are generally proportional to LOC, not number of deploys - given that's a DevOps variable and not a programming one – so the only relevant difference I can imagine is less chance of a multi-bug, multi-factorial, multi-dependent, nightmarish-to-fix incident.

Haha, fair enough.

> not that they were wearing a clan hat to work

If so, I'd go with "refer them to Uber or Coinbase" perhaps..

If it's 'bold and flagrant', then probably "don't hire them" would be a pretty good rule.

This is absurd. Don't entrust an online service with the only copies of important data if you don't want to lose it. Even in the most statist society imaginable, making it a criminal act to 'fully ban' a user from a 'service' is barely even definable, let alone feasible or desirable.

Yeah, you never know if the backup is really working, or if the deletion might (in some obviously-deeply-flawed logic/architecture) get streamed to the replica/backup, etc. I'm not taking a position on the workplace ethics issue, but from a sheerly technical standpoint I would not characterise that as a safe course of action, like that person seemed to be representing it.

Indeed, as are the graphics we're discussing.

Ah, sorry for the late response.

Respectfully, I think it's a bit dishonest that you've linked to one very specific subpage for one specific office in Seattle, presumably the one page you found which contained a lot of white people.

I'm not sure if I even need to write any more, because the fact that you needed to do that demonstrates my point probably better than I can.

But, to say the obvious, have a look at their root careers page (I think that's a good "nothing-up-my-sleeve URL", so to speak): https://careers.google.com

> To put what charitably?

Well, the phrase immediately surrounding that one in my comment: "their marketing graphics bely those [i.e. 53%-white] demographics".

(To be clear, I don't say this with any racist intent. I'm just allergic to insincerity. Whatever one's political commitments, it's plain – to anyone who hasn't completely subordinated their critical faculties to some or other ideological dogma – that Google's photos make great play of their few minority ethnic staff, more than would an equally-large set of random samples from their 53%-white staff. It reminds me of my sister (who's mixed-race, as I am) being summoned whenever her china-white private girls' school was taking photos. My own - hers was our sister school - mustn't've deemed me sufficiently dark..)

Ah, right, thanks! That wasn't actually part of the statement they quoted (/posed within quotation marks), whence my confusion.

Edit: Or is the point that 20% of floor workers in total do become managers, but that's precisely (or very strongly correlatedly) the subset containing those who have degrees? If so, that makes a lot more sense. I don't really think it's fair to describe as a fallacy, though - it's a function of the philosophy of probability, which doesn't describe the world but our knowledge of it. You could make an analogous statement about roulette wheels, with perfect knowledge of their physics.

Hmm, there were some absurdities when I was on the team, but I wouldn't have identified that as one of them. I'd characterise the service count as possibly slightly overboard, but not the deploy count: making changes super-incrementally was absolutely a positive thing compared with other places I've worked, although it would never work without an almost-zero-friction deployment process.

Done right, it avoids bugs - especially complex hard-to-roll-back state-related bugs - that can arise from discrete waterfall-style releases of big and complex features all at once.

Your comment seems to be making a hell of a lot of assumptions and associations which seem very specific to the environment you work in[0], but stated as though you think the entire world must be working in the same way. It feels a bit like a child insisting that they don't speak with an accent.

[0] "One deploy is one feature", "every company has 'BA's and 'PM's" [we had the latter but I'm scare-quoting both because they are far from universal], "each feature has to be 'identified' by said BA or PM", etc. Also, this seems to be written not only from a staid perspective but from a small-co perspective; for large companies, even 3000 features is not a huge number, and wouldn't be more than maybe 5 or 10 per team.