jvans

How do you personalize to the specific signals of the product, do they ingest into your infrastructure? What happens if a customer discovers a bug in a feature they're ingesting, how do they have control of retrains/pinning model versions? Who handles monitoring, the customer or your service?

The desire to rewrite something is the single biggest red flag for me that someone has questionable technical decision making skills. Yes there can be good reasons for it, but my priors shift dramatically once I hear someone suggest it

They mean rewriting. People love rewriting stuff they didn't write

why is a migration such a burden in that scenario

> Human beings, especially in groups, can only reasonably make linear decisions.

This seems to be getting a lot of attention. I couldn't agree more, we assume linearity all the time because reasoning non-linearly is exceptionally difficult. Yes we can do it sometimes, but it is not the default. Reasoning linearly has its flaws, and we should recognize we are making an imperfect decision, but it is still extremely useful.

> Yes, I will waste 45 minutes of my time explaining word for word to you, exactly what is written in this design document

And they will misunderstand you, turn around and butcher the explanation to some higher up, indirectly claiming credit for your work as the messenger of its result. If something goes wrong with the project, they'll definitely make sure to emphasize your involvement in the failed initiative, strategically distancing themselves.

This is just being hindsight oriented. The way it worked out it was better to DRY from the start, but the person implementing it didn't know how the chips would fall.

If you do what this guy did and are wrong, it looks like yes obviously it was the same thing and you should have DRY right from the start.

If you DRY from the start and are wrong, someone writes a blog post about not DRYing from the start.

It's a bet on which mistake is worse, because you will be wrong sometimes. DRYing from the start IMO is a worse mistake than duplication.

The problem with not null is people classify them from a domain perspective and not from a distributed systems perspective where basically everything can be null but only some fields mean rendering the page is impossible. It also depends page by page what fields make it unrenderable

Additional graph pain points I'd add:

Reliability:

* Not null fields in a distributed system are a lie. Clients write code assuming something is not null, so a query that fetches data from N sub systems breaks the entire page when one of them fails.

Performance:

* People say you only need to fetch just what the page wants but in practice clients create re-usable fragments for every object in the system and use it everywhere. Any time a new field is added, every api call fetches that new field adding latency everywhere

* clients are unaware of the underlying topology and add fields that are usually harmless but have bad tail latencies.

* The completely generic nature of the API means the backend can't optimize performance for a specific page. E.g. if one API has nasty tails, but it's not that important for this page, the backend could time it out at some reasonable value instead of holding up the entire request

the cost is usability. When there's dozens of options you don't need or care about, you have to spend the time to understand what they are to realize you don't need them

This is classic feature bloat of a product that's been around for a while. Some customer wants a really niche feature so product adds a new option for it. It's just one new option so the experience isn't that different. Rinse and repeat hundreds of times and nobody stops the think of the cumulative effect on the 99% of customers who don't need those features.

the compensation structure in big tech makes people lose their minds optimizing for that next promotion/stock grant. It helps big tech retain talented people but the incentives to look "impactful" drive all the wrong behaviors

small things don't show up on p&l reports or a/b tests so the bean counters ignore them. But they have an outsized impact on long term success and people who are really great at what they do sweat the small stuff regardless of the numbers

That's sort of true but also used as an excuse for crappy design/execution.

Raft is a lot easier to digest than paxos: https://raft.github.io/raft.pdf

Approaching the problem of doctor burnout by spamming patients with half correct responses to important medical questions is not a solution

Notice how all the benefits they mention are for physicians and not for the patients. That's a pretty backwards approach to healthcare

A lot of things I've seen engineers do in the name of cleaning up technical debt actively make things worse

Left to their own devices a lot of engineers will go to enormous lengths for either marginal improvements or changes that actively hurt. I can understand why leadership is skeptical of these investments because even for someone extremely technical, if you're not in the weeds everyday of that particular problem, it's very hard to know what's worth it

So is the implication here that any system that allows SSH and contains this malicious code is vulnerable?

Exactly. A lot of us work at #2 so we wish our predecessors saved us our current pain. But if they went that route we wouldn't be employed at that company because it wouldn't exist

Not all engineers care about engineering or quality. Boeing self selects for those that don't

Thank you for the thoughtful response! I have actually read Extreme Ownership. I agreed with most of it, although one thing I started noticing is what I'd call "fake ownership". Once people realize the value of ownership, they try to game the system and use those tools to manipulate. Boeing CEO's response to the latest disaster comes to mind. He claims "this is our fault, we'll fix it". But it's hard to take it at face value when you know the story. I guess like most aspects of human behavior, there's an element of earnesty that needs to be present.

Have you read any of the books on rickover? I can find three:

1. Rickover and the Nuclear Navy

2. The Rickover Effect: How One Man Made a Difference

3. Nuclear Navy, 1946-1962

Would love to know if you have a recommendation or anything else on rickover you'd recommend reading. This guy seems worth studying

I love this whole speech. I'm curious how people put some of these efforts into practice, particularly holding people to high standards while still giving them autonomy and ownership.

Holding someone to a high standard seems like it requires frequently telling them that their work isn't good enough and they need to refine or redo it in some major way. How can people feel ownership while they're being frequently told their output isn't satisfactory? How do you balance firing people who can't perform without creating a culture of fear?

This is the opposite of an MBA answer. MBAs are famous for hand waving over important details and this answer hammers home the point of them

Wouldn't low performers hurt production output and piss off upper management? What's the "ideal" employee who succeeds in that environment look like in their day to day work?

There is no single or set of standards that can prevent you from having to think critically about a result. Rarely are results so overwhelmingly positive that it's obvious what the result means. In those cases it doesn't matter what method you use anyway. The rest of the time, you should use many methods, be wary of their assumptions and think very hard about what the data is telling you. Realize that you are biased in favor of a positive result and play devil's advocate with yourself. This is very very hard work, and most people don't want to do it, especially when their careers improve when they look the other way

in my experience academics are the worst at understanding uncertainty. They refuse to believe anything that doesn't pass their beloved t-test, believing the holy threshold of 0.05 is the arbiter of all truth

This is exactly what you should expect. Documentation is corporate frauds biggest enemy. If you are going to commit crimes and kill people, it's far better that you don't have records of incompetence even if lack of records is itself a display of incompetence.

Official logs of problems can give you insight into how deep rooted and severe the issues are. Lack of documentation means there are problems but allows them to hide the depth of them. E.g. we can't know how many other door plugs were shipped without bolts

And Boeing of course knows this, and this is exactly why they're so brazenly ignoring NTSB requests.

I would settle for major booking sites to provide a clear filter for Airbus. Not everyone will use it but it'll go a long way for those of us that do care