Hacker News Reader

In a very real way, it's all conditional jumps in assembly, and every thing you've learned to make programming easier by allowing more directly letting you express your high level intent is just sugar. It might even help some or most of the time. But what you're actually doing is creating a bunch of branches and loops, and as much as the high level stuff might help, you really shouldn't forget this is the medium you actually work in.

Most professions have a healthy respect for the base materials they work with no matter how high the abstractions and structures they build with it go. Artists know their paints, stone, metal, etc. Engineers know their melaterials as well. They build by taking the advantages of each material into consideration, not assuming that it's no longer relevant to their job because they get to just work in I-beams. Programmers would do well to adopt a healthy respect for their base materials, and it seems like often we don't.

Agree with this a lot. In other words, don’t be too clever. That leads to an unmaintainable codebase. There is value in simplicity and not overly using abstractions that take you farther and farther away from bare metal.

> There is value in simplicity and not overly using abstractions that take you farther and farther away from bare metal.

This is a contradiction. Simplicity is obtained through abstractions. As an example, fifty years ago, 'goto' reigned supreme. Then structured 'if/else/for' took over, as the superior abstraction over 'goto'. Now use of 'goto', while being far simpler to implement and closer to the bare metal, is commonly derided by many programmers.

The long term trend in software is constantly increasing abstraction hiding mountains of complexity, which increases both simplicity and complexity. Writing print('hello world') is simple, but the compiler, OS, silicon, etc. that makes that simplicity possible is extremely complex.

> high level intent is just sugar

I disagree with how your use of "just" here. It's common for programmers to dismiss the importance of syntax but syntax and notation are the interface and UX between the language semantics and your brain. It's no less important to get this right. There's a half-joke that Europe was able to rapidly advance in Calculus beyond Britain due to the superiority of Leibniz notation.

> healthy respect for their base materials

What's unique about computers is the theoretical guarantee that the base does not matter. Whether by lambda calculus, register machines or swarms of soldier crabs running from birds in specially designed enclosures, we're fine as long as we appropriately encode our instructions.

> bunch of branches and loops

You could also easily say it's just a bunch of state machines. We outsource tedious complexity and fine details to compiler abstractions. They track things for us that have analogues in logical deduction so that as long we follow their restrictions, we get a few guarantees. When say, writing asynchronous non-deterministic distributed programs, you'll need all the help you can get.

Even designing close to the machine (which most programs will not need) by paying attention to cache use, memory layout, branch divergence or using SIMD remain within the realm of abstractions.

Programming is primarily about managing complexity. Other than perhaps mathematics, there is no other field that must, should and can apply the amount of abstraction on top of abstraction as software engineers do.

It is a must, because decades of business requirement built on top each other without understanding the whole is complex. Writing a JIT-compiler that can routinely change between interpreting code and executing it natively, a database optimizing queries, a mathematical library using some fancy algorithm are all complex, in a way that is not reducible.

Complexity easily outgrowth even the whole of our mathematics, we can’t prove any non-trivial property of a program, halting problem, etc.

So all in all, no, we can only respect our “base materials” by finding the proper abstraction for the problem, as our base material is complexity itself. It might be for loops and ifs, but it very well be a DSL built on top of who knows how many layers, because at that abstraction level can we even start to map the problem domain to human consumable ideas.

> Personally I disagree. We should be using state machines and pure functions. If+for loops are just what's easiest to express in the major languages of today. they are no more or less computationally expensive but due to lack of tooling they are often cheaper to write.

In my experience programming programming with primitives and basic flow control operations frequently tends to be at least be order of magnitude faster than more complex state management paradigms. Compilers are very good at optimizing that style of code. Loops often get unrolled, the and the branch predictor is kept happy. A good compiler may use vector expressions.

In many cases with cold code it flat out doesn't matter, the CPU is plenty fast, but when it does matter, explicit if-and-for code absolutely mops the floor with the alternatives.

for loop goes brrrrrrrrrrrrrrrrrrr!

At least in the C++ space, stuff like boost-sml is able to produce assembly that is often as fast or occasionally faster than handwritten if or switch based FSM logic.

Will your manually written imperative code beat an SQL database for the same task? Because it uses a very very high level description on what it has to do and chooses an appropriate algorithm for that for you.

You can optimize one specific query (quite painstakingly, I believe) to beat a general db, but it is very far from clear that “for loops will beat higher level abstractions”, imo.

In designing my search engine, I ended up building a custom index instead of using an off-the-shelf database solution. Franky it's so much data a SQL database can't even load it when run on the same server that runs my entire search engine.

With my own set-up I can have my data in pre-calculated immutable tables that are memory mapped off disk. That means I can write lock-free code. My updates are large hour-long batch operations off a written journal.

For the lexicon I'm also rolling a special one-way compression scheme/hash function that guarantees uniqueness but only works due to quirks with the data.

General purpose databases can't do these things, because then they wouldn't be general purpose databases.

SQL databases perform really well on average given your query is relatively optimized, but they do not represent the pinnacle of performance. Any time you want the fastest most efficient code to deal with a particular use case, it pays enormous dividends to roll your own implementation that has domain awareness. It means you can cut corners general solutions simply can't.

FSM and pure functional kernel based designs you can get just as clear logic that is expressible not just to the programmer but also to business personnel

I’m yet to see a secretary who could “return a new table state such that as if documents became a right fold as binding together a map of composition of signing and copy routines over documents” instead of “get these documents from the table, copy, sign and put them back in a binder”. This is a nonsense some of us want to believe in, but it is not true.

I inherited a piece of code that was designed as a state machine and the state machine design basically made the project become pure technical debt as requirements increased over time and engineers had to make it work with the state machine model that had been chosen when the project started.

If the project had instead been designed to have less unnecessary state and “transitions” it would have been a lot easier to make changes.

All those ideas sound good by themselves but they are really bad for “defensive” coding. Good luck selling a project to refactor something when it’s going to take multiple people-years. Once you’ve made the mistake of committing to an inflexible design it’s either that, replace/ignore the thing, or deal with low productivity as long as the thing exists.

> state machine model that had been chosen when the project started.

so was the chosen model the issue or choosing a state machine model at all?

State machines are refactorable if you use state charts. Using one giant FSM is like committing compiled code.

I've written many SM implementation starting from one used in low protocols and up to business process middleware so I have an experience and know how incredibly useful and powerful those are when used in right place. But to use them everywhere especially in some math algos would be an insanity worse than GoTo.

>They are well understood and most people can understand them with little to no additional education past what they learned in their business or engineering bachelors degrees.

Imperative code:

    Take A and B.
    Add C ml of Water.
    Stir for 2 minutes.
    If it thickened, add D grams of flour, else go back to stiring.

This is easily understood by everyone, degree or no. Once someone figures out loops and the difference between statement and expression, they can essentially understand imperative code.

Okay, now put boxes around each sentence and connect them with arrows. ;-)

Imperative code quickly devolves to a point where it is no longer understandable by anyone but the developers who wrote it.

> We should be using state machines and pure functions.

For problems where those are the right tools, sure. But they aren't the right tools for all problems any more than ifs and for loops are.

I've always felt like explicit state machines are the sledge hammer you break out when you can't find any good abstraction to encapsulate the logic. As an intermediate step for parsers it's pretty powerful, but it's not something I want in my hand written code if I have any alternatives.

do you know some nice examples of medium complexity that you can show for inspiration?