edejong

On point

In 2000 we organized a large outdoor LAN party for a week in the middle of nowhere hosting around 600 Unix/Linux aficionados. I was very proud to have found Bram to speak there at our event. On the days ahead I became very anxious to welcome this paragon of open source contributors, but I soon found out Bram was one of the most fun and caring people I had the honour to meet. Not just did he tell us his story of developing vim, he also organised an in promptu quiz on Unix and networking. I have fond memories of Bram and am truly sad to hear he passed away. Bram truly captured the soul of open source.

I wish we rephrased stuff by mentioning actors:

“Our careless actions have resulted in an unprecedented and alarming heatwave in the sea.”

Really curious: Half of the code base in LoC or in functionality?

A good programmer knows the limits of the type system. They lean on that knowledge to determine which tests to write.

He’s not acting toxic. You just proved to have a thin skin.

Luckily Popper explained to us we have something in science called falsifiability. I suggest you try to falsify one single theory of our current climate science. Or, alternatively, show that any of the theories is unfalsifiable. Just going all Popper on you, because it’s needed.

By the way, much of our previous research on climate science has already been falsified, for them to be substituted for theories which show even stronger evidence for man-made climate change.

I understood the article just fine. The "Let It Crash" philosophy is scale invariant.

Please read the last three paragraphs in [0]: "a well-designed application is a hierarchy of supervisor and worker processes" and "It handles what makes sense, and allows itself to terminate with an error ("crash") in other cases."

I've personally designed and co-implemented mission critical real-time logistics systems which dealt with tens of thousands of events per second, with hundreds of different microservices deployed on a cluster of 14 heavy nodes. Highly complex logic. At first we were baby-sitting the system, until it became resilient by itself. Stuff crashed all the time. Functional logic was still intact. Then we had true silence for months on our pager alerts.

I call it anti-fragile and Taleb is right. You can't make a system resilient if you don't allow it to fail.

[0] https://wiki.c2.com/?LetItCrash

Let it crash is anti-fragile. Either you understand that, or you try to make super robust and super expensive applications.

In the 27 minutes after you wrote your comment, our CI/CD rebuild two containers from a latest snapshot. No manual work involved.

CPU/Memory: scale horizontally when needed. Monitor cost.

Disk: essentially limitless. If disk of VM runs out, node will crash, new node will start. Service should keep running on other nodes meanwhile. If restarts happen too often, throughput/error rate will suffer or cost will rise.

Modern mentality is "let it crash" [0]. Software has bugs. Design so it can crash and scale depending on need.

[0] https://medium.com/@vamsimokari/erlang-let-it-crash-philosop...

* k8s helps because you keep your application online during rolling updates. You keep your changes relatively small and can test your changes in isolation.

* microservices help security because for each service you write, you create a minimal surface area (as opposed to monolithic applications where each method has access to maximum dependencies).

* microservices also help monitoring, because failures can be contained and one application does (should) not impact another. This means that your focus of monitoring is on application performance, and less on the underlying system architecture.

* monitoring can be a cross-cutting concern. For example, you can monitor all HTTP requests by instrumenting your ingress layer. Similarly, you can support a unified authorisation scheme by bringing this to the ingress layer. Even more, using microservices, you can independently promote services to production (sort of like feature toggles) or do green/blue promotion.

* security of your microservices is improved, because you can independently upgrade your dependencies per service. Many older monoliths are 'stuck' in a certain dependency configuration and cannot be upgraded without a 'big bang' of expensive dependency resolution and testing.

I could give many more examples, but I hope this captures the spirit a little.

In the meantime, your PHP app had 40 major security flaws, no meaningful monitoring, a DB that wasn't backed up and major data consistency problems. Also, when your box's hdd crashed, you lost all those minor changes you had vim'ed over the years. But for the rest, yeah, all is fine.

Remastered often equals compressed to the max. It means that soft passages become as loud as the loud passages. Read about it more by googling 'loudness war'.

I totally agree. JSON at the application level is too simple of an abstraction, leading to unnecessary complexity.

Absolutely. The lasting ideas are KISS and Occam's Razor.

But you know what happened to SOAP? It was smothered by its own success. The allure of interoperability and extensibility attracted everyone and their cat to the party. With so many stakeholders involved, it became a classic case of "too much design by committee." It's like trying to paint a masterpiece by having a thousand artists contribute strokes—chaos ensues.

And let's not forget the influx of junior developers during that time. I mean, we can't blame them entirely, can we? SOAP standards were complex and enormous. It's no wonder they struggled to grasp the underlying paradigms. We had an army of fresh faces flooding the scene, and the sheer complexity overwhelmed them.

So, SOAP ended up being the baby tossed out with the bathwater. It had its merits, but the challenges it faced were just too much to bear. Still, it's worth reflecting on its strengths and the lessons we can learn. Maybe someday we'll find a way to strike a balance between the elegant core idea and the practical realities of implementation.

As I think about all the technologies that have come and gone, it's good to find some sturdy rocks as well. Here are some techs that withstood the test of time in my personal journey:

* Simple, but reliable file formats: My old photos collection is 25 years old (jpeg) and I still look at them occasionally.

* My influxdb/grafana/openhab setup is now 8 years old and has been operational with only minimal hickups.

* Some home automation scripts I wrote have endured for 8 years now.

* My Linux/Unix knowledge is 26 years old (although much has changed since Slackware).

* My C and C++ programming language knowledge is 28 years old (although I rarely use it anymore, because of Rust). Occasionally it is useful during debugging performance problems in kernels.

* My knowledge of distributed systems (stuff like leader election, distributed and centralised algorithms) is 15 years old. It is still very relevant to understand both existing systems and to evaluate new systems.

It is crucial to distinguish the knowledge that stands the test of time from the fleeting trends. And finally then there are the non-technical skills, which pay off even more. Effective communication, negotiation skills, organisational finesse, inspirational leadership, adept management... These are the true constants that support us during our professional life.

If an IP-address is PII, then so should be a VIN and chassis number.

I prefer olive oil butter over any margarine. For almost all applications, I prefer it over butter.

We decided it’s better to present this in a curated program called education.

It seems the whole research community is in flux.

Yes, that’s an interesting topic, especially given the prevalence of distributed compute nowadays and the rising awareness of cloud costs.

In the end, the distribution is not really poisson, of course. So, you might be interested in low pass filtering to elastically scale your provisioned workers. There is quite some theory about this, including sophisticated machine learned models to predict future load. But I digress.

What are spike workloads? Is this poisson? What’s the distribution?

True, my example is an oversimplification. In reality you want to account for rapidly and slowly changing demands. This can be solved with elasticity of systems or by overprovisioning. You can get deep, with time series analysis, low pass filtering and various metrics, such as quantile distributions… however, saying zero queue length is desirable is a gross oversimplification.

Poisson. And 1.0 variance in request processing time.

Queues should have an ideal size based on your latency requirements.

Say, you have 100 requests coming in per second. Each consumer of your queue requires 0.3 seconds per request. Now, you can optimise for the number of consumers. Would you choose 40 consumers, then the probability of serving the request with an empty queue in 0.3 seconds would be 17% (and > 0.3 seconds would be 84%). However, the total waiting time per request would be 0.1 second.

With 80 consumers, the probability of an empty queue would be higher at 58%. However, most of those consumers are doing nothing, while your waiting time is only reduced by 0.1 second! Only a 30% improvement.

(edit: probability of empty queue in second example is higher, not lower)

Queues should absolutely not be empty. If they are often empty, you're over provisioning your consumers. There is an ideal size to a queue, based on the balance equation (https://en.wikipedia.org/wiki/Balance_equation). Also study queueing theory if you want to know more: https://en.wikipedia.org/wiki/Queueing_theory.

Required.

At a Dutch CS study we had an SQL course that started with first-order logic, relational algebra and went to on to project that into SQL. It also taught 3NF/4NF and BCNF, indices, r-trees, query planning and optimisation.