movpasd

"Worse is better" remains relevant as ever.

https://www.jwz.org/doc/worse-is-better.html

Frustratingly, many of the questions have multiple answers that can apply simultaneously! (You may like to guess my result.)

There's a narrative that gets passed around in physics circles about how the Soviets were better at finding creative and analytical solutions than Americans, because of the relative scarcity of computing versus intellectual labour resources.

It would make sense to me that a parallel mechanism could apply to Soviet engineering. If material and technologically advanced capital are scarce, but engineers are abundant, you would naturally spend more time doing proper engineering, which means figuring out how to squeeze the most out of what you have available.

I prompt politely for two reasons: I suspect it makes the model less likely to spiral (but have no hard evidence either way), and I think it's just good to keep up the habit for when I talk to real people.

Regarding LLM's performances on brownfield projects, I thought of Naur's "Programming as Theory Building". He explains an example of a compiler project that is taken over by a team without guidance from the original developers:

> "at [the] later stage the original powerful structure was still visible, but made entirely ineffective by amorphous additions of many different kinds"

Maybe a way of phrasing it is that accumulating a lot of "code quality capital" gives you a lot more leverage over technical debt, but eventually it does catch up.

Type systems, database normalisation, reactive programming (i.e.: non-manual state management), or more generally making illegal states unrepresentable are the software equivalents!

If your required logic separates nicely into steps (like "fetch, compute, store"), then a procedural interface makes sense, because sequential and hierarchical control flow work well with procedural programming.

But some requirements, like yours, require control flow to be interwoven between multiple concerns. It's hard to do this cleanly with procedural programming because where you want to draw the module boundaries (e.g.: so as to separate logic and infrastructure concerns) doesn't line up with the sequential or hierarchical flow of the program. In that case you have to bring in some more powerful tools. Usually it means polymorphism. Depending on your language that might be using interfaces, typeclasses, callbacks, or something more exotic. But you pay for these more powerful tools! They are more complex to set up and harder to understand than simple straightforward procedural code.

In many cases judicious splitting of a "mixed-concern function" might be enough and that should probably be the first option on the list. But it's a tradeoff. For instance, you then could lose cohesion and invariance properties (a logically singular operation is now in multiple temporally coupled operations), or pay for the extra complexity of all the data types that interface between all the suboperations.

To give an example, in "classic" object-oriented Domain-Driven Design approaches, you use the Repository pattern. The Repository serves as the interface or hinge point between your business logic and database logic. Now, like I said in the last paragraph, you could instead design it so the business logic returned its desired side-effects to the co-ordinating layer and have it handle dispatching those to the database functions. But if a single business logic operation naturally intertwines multiple queries or other side-effectful operations then the Repository can sometimes be simpler.

You might this blog post interesting, which argues that it's Rust semantics and not syntax that results in the noisiness, i.e.: it's intrinsic complexity:

https://matklad.github.io/2023/01/26/rusts-ugly-syntax.html

I found it reasonably convincing. For what it's worth, I found Rust's syntax quite daunting at first (coming from Python as well), but it only took a few months of continuous use to get used to it. I think "Perl-esque" is an overstatement.

It has some upsides over Python as well, notably that the lack of significant whitespace means inserting a small change and letting the autoformatter deal with syntax changes is quite easy, whereas in Python I occasionally have to faff with indentation before Black/Ruff will let me autoformat.

I appreciate that for teaching, the trade-offs go in the other direction.

I think the fundamental issue with implementation-inheritance is the class diagram looks nice, but it hides a ton of method-level complexity if you consider the distinction between calling and subtyping interfaces, complexity that is basically impossible to encapsulate and would be better expressed in terms of other design approaches.

With interface-inheritance, each method is providing two interfaces with one single possible usage pattern: to be called by client code, but implemented by a subclass.

With implementation-inheritance, suddenly, you have any of the following possibilities for how a given method is meant to be used:

(a) called by client code, implemented by subclass (as with interface-inheritance) (b) called by client code, implemented by superclass (e.g.: template method) (c) called by subclass, implemented by superclass (e.g.: utility methods) (d) called by superclass, implemented by subclass (e.g.: template's helper methods)

And these cases inevitably bleed into each other. For example, default methods mix (a) and (b), and mixins frequently combine (c) and (b).

Because of the added complexity, you have to carefully design the relationship between the superclass, the subclass, and the client code, making sure to correctly identify which methods should have what visibility (if your language even allows for that level of granularity!). You must carefully document which methods are intended for overriding and which are intended for use by whom.

But the code structure itself in no way documents that complexity. (If we want to talk SOLID, it flies in the face of the Interface Segregation Principle). All these relationships get implicitly crammed into one class that might be better expressed explicitly. Split out the subclassing interface from the superclass and inject it so it can be delegated to -- that's basically what implementation-inheritance is syntactic sugar for anyway and now the complexity can be seen clearly laid out (and maybe mitigated with refactoring).

There is a trade-off in verbosity to be sure, especially at the call site where you might have to explicitly compose objects, but when considering the system complexity as a whole I think it's rarely worth it when composition and a tiny factory function provides the same external benefit without the headache.

These are powerful tools, if used with discipline. But especially in application code interfaces change often and are rarely well-documented. It seems inevitable that if the tool is made available, it will eventually be used to get around some design problem that would have required a more in-depth refactor otherwise -- a refactor more costly in the short-term but resulting in more maintainable code.

You can either use the @example decorator to force Hypothesis to check an edge case you've thought of, or just let Hypothesis uncover the edge cases itself. Hypothesis won't fail a test once and then pass it next time, it keeps track of which examples failed and will re-run them. The generated inputs aren't uniformly randomly distributed and will tend to check pathological cases (complex symbols, NaNs, etc) with priority.

You shouldn't think of Hypothesis as a random input generator but as an abstraction over thinking about the input cases. It's not perfect: you'll often need to .map() to get the distribution to reflect the usage of the interface being tested and that requires some knowledge of the shrinking behaviour. However, I was really surprised how easy it was to use.

I suspect Minecraft was large enough to support an effective modding community from the start regardless of official support, so that there was always some kind of third-party unofficial mechanism (ModLoader, then Forge, then now Fabric and Quilt). Mojang probably punted it down the priority list because of that, or didn't want to impose a structure and kill those ecosystems. Technically speaking, Java is reasonably easy to plug stuff into at runtime, so that was never a barrier.

The original issue with official modding support, from my perspective, has always been a legal one. But the Mojang EULA explicitly allows modding now. So I would see this decision as one in a long line of decisions by Mojang to both normalise the legal relationship with modders, and beyond that giving a "thumbs up" to the community.

You may be interested in Mojo, it's a project by Chris Lattner. It aims to have Python-like syntax and smooth integration with Python but allow Rust-like low-level control (I believe it has a borrow checker). Unfortunately, I believe it's proprietary.

I think this is a paradigm known as concatenative programming: https://en.wikipedia.org/wiki/Concatenative_programming_lang...

It may be a combination of both: the fact it is easier to stay in touch makes it more difficult to let go of friendships. But this may make those friendships feel less meaningful and therefore increase loneliness.

I used Claude to get the outage start and ends from the post-event summaries for major historical AWS outages: https://aws.amazon.com/premiumsupport/technology/pes/

The cumulative distribution actually ends up pretty exponential which (I think) means that if you estimate the amount of time left in the outage as the mean of all outages that are longer than the current outage, you end up with a flat value that's around 8 hours, if I've done my maths right.

Not a statistician so I'm sure I've committed some statistical crimes there!

Unfortunately I can't find an easy way to upload images of the charts I've made right now, but you can tinker with my data:

    cause,outage_start,outage_duration,incident_duration
    Cell management system bug,2024-07-30T21:45:00.000000+0000,0.2861111111111111,1.4951388888888888
    Latent software defect,2023-06-13T18:49:00.000000+0000,0.08055555555555555,0.15833333333333333
    Automated scaling activity,2021-12-07T15:30:00.000000+0000,0.2861111111111111,0.3736111111111111
    Network device operating system bug,2021-09-01T22:30:00.000000+0000,0.2583333333333333,0.2583333333333333
    Thread count exceeded limit,2020-11-25T13:15:00.000000+0000,0.7138888888888889,0.7194444444444444
    Datacenter cooling system failure,2019-08-23T03:36:00.000000+0000,0.24583333333333332,0.24583333333333332
    Configuration error removed setting,2018-11-21T23:19:00.000000+0000,0.058333333333333334,0.058333333333333334
    Command input error,2017-02-28T17:37:00.000000+0000,0.17847222222222223,0.17847222222222223
    Utility power failure,2016-06-05T05:25:00.000000+0000,0.3993055555555555,0.3993055555555555
    Network disruption triggering bug,2015-09-20T09:19:00.000000+0000,0.20208333333333334,0.20208333333333334
    Transformer failure,2014-08-07T17:41:00.000000+0000,0.13055555555555556,3.4055555555555554
    Power loss to servers,2014-06-14T04:16:00.000000+0000,0.08333333333333333,0.17638888888888887
    Utility power loss,2013-12-18T06:05:00.000000+0000,0.07013888888888889,0.11388888888888889
    Maintenance process error,2012-12-24T20:24:00.000000+0000,0.8270833333333333,0.9868055555555555
    Memory leak in agent,2012-10-22T17:00:00.000000+0000,0.26041666666666663,0.4930555555555555
    Electrical storm causing failures,2012-06-30T02:24:00.000000+0000,0.20902777777777776,0.25416666666666665
    Network configuration change error,2011-04-21T07:47:00.000000+0000,1.4881944444444444,3.592361111111111

Seems like an example of "worse is better". The worse solution has better survival characteristics (on account of getting actually made).

I believe this is called epistemic pragmatism in philosophy: https://en.wikipedia.org/wiki/Pragmatism

This is covered in the article. The uncertainties brought about by zonal pricing are not really worth it, given that the main obstacle is the need for network reinforcement. The UK is just not that big! Introducing a complicated market reform which will be obsoleted within a few decades doesn't make sense.

Everyone's different. Some people genuinely thrive under the conditions you're describing, others don't like it but are able to put up with it no problem, and others can't stand it but are forced to.

The perspective I've found most useful is this. There is a constellation of correlated "autistic traits", which everyone has to a degree, but which like most traits become disabling if turned up too much. "Autism" is a term describing that state. So, it is much less a particular switch that can be turned on or off, not even a slider between "not autistic" and "very autistic", but more a blurry region at the outskirts of the multidimensional bell curve of the human experience.

People on the furthermost reaches of this space are seriously, unambiguously disabled, by any definition. They're what people traditionally imagine as "autistic". But the movement in psychiatry has been to loosen diagnostic criteria to better capture the grey areas. Whether this is a good or a bad thing is a social question, not a scientific one, in my opinion. Most of us want to live in a society that supports disabled people, but how many resources to allocate to that is a difficult question where our human instincts seem to clash with the reality of living in a modern society.

On your last paragraph: I think this is a serious problem with the discourse around neuroatypicality today. My opinion is that the important thing is that we become more accepting and aware of the diversity of the human experience, and that this is a necessary social force to balance the constant regression to the mean imposed by modernity. If that's the case, then drawing a border around any category of person, staking a territorial claim to a pattern of difficulty the group experiences, and refusing to accept that the pattern exists beyond it: it's just unfair, it's giving into defensiveness.

For myself, I have low energy if I don't eat breakfast, but there is essentially no hunger signal for me in the morning. Over time I've settled on eating the plainest breakfast I can.

I think this has a lot to do with the 9–5 and my natural sleep cycle being delayed compared to that.

I think this comes back to the idea of having a "UX model" that underlies the user interface, laying out its affordances clearly in code. In a modern application you're going to have complex UX logic and state that's distinct from the domain model as such and that deserves representation in the code.

In an MVC conception, the UX model becomes a top layer of abstraction of the domain model. It's a natural place to be because for modern apps, users expect "more than forms", i.e.: different ways of cutting up the domain data, presented in different ways, ...

This is something that component-based frontend frameworks struggle with a bit: the hierarchical layout of the DOM doesn't always reflect the interrelations in data between parts of a user experience. Prop drilling is just a reflection of this fact and perhaps it's why we're seeing a rise in the use of state stores. It's not really about state, that's just the technical symptom, it's really about providing a way of defining a (in-browser) data model based on the user experience itself rather than the particularities of the UI substrate.

At the risk of sounding woo, I find some parallels in how LLMs work to my experiences with meditation and writing. My subjective experience of it is that there is some unconscious part of my brain that supplies a scattered stream of words as the sentence forms --- without knowing the neuroscience of it, I could speculate it is a "neurological transformer", some statistical model that has memorised a combination of the grammar and contextual semantic meaning of language.

The difference is that the LLM is _only that part_. In producing language as a human, I filter these words, I go back and think of new phrasings, I iterate --- in writing consciously, in speech unconsciously. So rather than a sequence it is a scattered tree filled with rhetorical dead ends, pruned through interaction with my world-model and other intellectual faculties. You can pull on one thread of words as though it were fully-formed already as a kind of Surrealist exercise (like a one-person cadavre exquis), and the result feels similar to an LLM with the temperature turned up too high.

But if nothing else, this highlights to me how easily the process of word generation may be decoupled from meaning. And it serves to explain another kind of common human experience, which feels terribly similar to the phenomenon of LLM hallucination: the "word vomit" of social anxiety. In this process it suddenly becomes less important that the words you produce are anchored to truth, and instead the language-system becomes tuned to produce any socially plausible output at all. That seems to me to be the most apt analogy.

I agree with the overall message, but I will say that there is still a great deal of value in memorisation. Memorising things gives you more internal tools to think in broader chunks, so you can solve more complicated problems.

(I do mean memorisation fairly broadly, it doesn't have to mean reciting a meaningless list of items.)

That's also my view. It's clear that these models are more than pure language algorithms. Somewhere within the hidden layers are real, effective working models of how the world works. But the power of real humans is the ability to learn on-the-fly.

Disclaimer: These are my not-terribly-informed layperson's thoughts :^)

The attention mechanism does seem to give us a certain adaptability (especially in the context of research showing chain-of-thought "hidden reasoning") but I'm not sure that it's enough.

Thing is, earlier language models used recurrent units that would be able to store intermediate data, which would give more of a foothold for these kind of on-the-fly adjustments. And here is where the theory hits the brick wall of engineering. Transformers are not just a pure machine learning innovation, the key is that they are massively scalable, and my understand is part of this comes from the _lack_ of recurrence.

I guess this is where the interest in foundation models comes from. If you could take a codebase as a whole and turn it into effective training data to adjust the weights of an existing, more broadly-trained model, But is this possible with a single codebase's worth of data?

Here again we see the power of human intelligence at work: the ability to quite consciously develop new mental models even given very little data. I imagine this is made possible by leaning on very general internal world-models that let us predict the outcomes of even quite complex unseen ("out-of-distribution") situations, and that gives us extra data. It's what we experience as the frustrations and difficulties of the learning process.

In Python 3.12 syntax, you can use

    type UserIs = UUID

I think it can be useful to think of the parsing and logic parts both as modules, with the parsing part interfacing with the outside world via unstructured data, and the parsing and logic parts interfacing via structured data, i.e.: the validated types.

From that perspective, there is a clear trade-off on the size of the parsing–logic interface. Introducing more granular, safer validated types may give you better functionality, but it forces you to expand that interface and create coupling.

I think there is a middle ground, which is that these safe types should be chunked into larger structures that enforce a range of related invariants and hopefully have some kind of domain meaning. That way, you shrink the conceptual surface area of the interface so that working with it is less painful.

I think you and the article are referring to abstractions over different concerns.

The concern you're talking about is about the actual access to the data. My understanding of the article is that it's about how caching algorithms can abstract the concern of minimising retrieval cost.

So in some ways you're coming at it from opposite directions. You're talking about a prior of "disk by default" and saying that a good abstraction lets you insert cache layers above that, whereas for the author the base case is "manually managing the layers of storage".

100k to 200k is a 100% increase in absolute, but a 10 percentage point increase to your target of 1M. The difference between the example you give and the one in the article is that 0 in the case of your pension meaningfully refers to its emptiness, but in the case of Celsius, it has no "emptiness" interpretation.

The equivalent would be saying that going from 600k to 700k was a 100% increase... compared to 500k.

It's not completely meaningless, to be fair. Saying 10°C to 20°C is a 100% increase has the meaning of "it's twice as far from freezing", which isn't totally meaningless (kind of like saying Everest is twice as high as Mont Blanc, which really means "its summit is twice as far from sea level").

The issue is a percentage of a Celsius value is not that. For example, an increase from 1°C to 2°C is a "100% increase", but is only 1 percentage point from freezing to boiling.

This is actually a deliberate design choice, which the breathtakingly short JSON standard explains quite well [0]. The designers deliberately didn't introduce any semantics and pushes all that to the implementors. I think this is a defensible design goal. If you introduce semantics, you're sure to annoy someone.

There's an element of "worse is better" here [1]. JSON overtook XML exactly because it's so simple and solves for the social element of communication between disparate projects with wildly different philosophies, like UNIX byte-oriented I/O streams, or like the C calling conventions.

---

[0] https://ecma-international.org/publications-and-standards/st...

[1] https://en.wikipedia.org/wiki/Worse_is_better