Hacker News Reader

Comment by valuearb | Hacker News Reader

"For Lamport, a major reason today’s software is so full of bugs is that programmers jump straight into writing code. “Architects draw detailed plans before a brick is laid or a nail is hammered,” he wrote in an article. “But few programmers write even a rough sketch of what their programs will do before they start coding.”

I almost always dive in, but I almost always write my code twice. Essentially the first round is my rough sketch, the second is written when I fully understand the problem, which for me only occurs once I've tried to code it.

View on HN

Exactly. This is ignoring the fact that code is a more useful blueprint than anything else for programs.

I'm sure if architects had the ability to magically conjure building materials out of thin air and try things in real life for free, architecture would involve a lot more trying and a lot less planning.

Of course, I'm sure the article is talking about people just rushing into production code without thought, and I get that is a problem. It's just when people make comparisons like that it can imply this horrendous future where we whiteboard program for months, which is just a terrible idea.

Coding isn't always the best blueprint. One example: what if you're writing a service that talks to two other internal services and a third-party API? Your code might capture what your specific service does, but it doesn't capture the overall design and intent of the complete system. That's one of the places where formal methods like TLA+ excel.

You might be interested in Servant, a haskell library that uses types to capture these high level interactions between components and then validate and even generate clients, documentation, servers, and simulators.

I definitely would! One thing a lot of people miss in these discussions is that this isn't an either-or, and we should be using multiple different correctness techniques to increase our confidence.

It's true that source code is a blueprint (we don't have builders; for software, unlike civil engineering, the build process is done by automated tools, not people.)

There's certainly an argument that coding is often done without clear understanding of the larger-scale structure that the module is going to fit into, the exact domain functionality, etc., and that much development has gone too far in avoiding analysis and requirements, overreacting to the broken and frontloaded abstract design process that often used to be done prior to writing any code.

But missing blueprints are the wrong analogy.

> This is ignoring the fact that code is a more useful blueprint than anything else for programs.

Personally, I find good 'ole bubble-and-arrow diagrams, list of data members, and hand-written C structs with arrows between them to be extremely valuable prototyping tools.

I think there's a bit of a problem with the architecture metaphor in general. It implies a limited range of things planned and built. As software eats the world and does more and more things, a metaphor that more closely reflects the variety of software is people telling other people to do stuff.

If you tell someone to bring you your socks, you don't need to plan for it. If they bring the wrong socks, you just fix it. If you tell them to invade the beaches at Normandy, you might want to work out more of the details in advance. You can tell someone to remove a splinter or remove a brain tumor, and your part of the instructions might be roughly equivalent if you are telling an "API" that has already been adequately told how to do what you ask.

The problem of unintended consequences of instructions has been with us far longer than computer software. In any story of a magic jini granting three wishes, the third wish always ended up a version of `git reset --hard`. I love having direct manipulation tools that simulate your proposed solution, giving you much faster feedback. Midas with VR goggles would have quickly seen something unintended turn to gold and canceled before committing. That's extremely helpful.

But this isn't the ultimate solution for how to deal with software complexity. It's a very helpful tool in some cases. Some software should still just be coded immediately and fixed as needed (takes less time to do it again than to create a simulator), some would benefit most from a good, debugged library (I'd rather the robot already know how to remove the tumor than show me direct feedback of my fumbling), some from direct manipulation tools, some from mathematical techniques (remembering that mathematically proven software is buggy if an errant RAM chip violates the axiom that `0=0`), some from better testing, some from better runtime monitoring, and so on.

But as with humans' verbal instructions, there will always be leftover unanticipated consequences due to flaws in the spec, bugs in the code, and breakage in the implementation.

One way the architecture metaphor breaks down is that very tiny details can have incredibly large knock-on effects in software. This is different from normal architecture and building engineering where it's certainly true that there are many details that need to be carefully considered, but those things are well understood and managed from project to project. Software on the other hand could be doing anything, in a world of pure logic traversing scales analogous to the smallest quark ranging to the entire universe. Building physical things just doesn't deal with those scales, you only have to worry about material properties and structures within a range of few orders of magnitude.

Apparently a "subtle conceptual error" can have massive consequences in architecture.

http://people.duke.edu/~hpgavin/cee421/citicorp1.htm

Oh dear. Thank goodness civil engineers do not build by trial and error. Many programmers do use the trial and error approach but their programs are riddled with bugs. There is no way that code is the best or most useful "blueprint" for software.

Your analogy is quite good, though, just not in the way you intended. If civil engineers did what you suggest then they would build magnificent structures which work in their office environment. Of course when that window that's never opened is opened by a cleaner everything comes tumbling down due to a gust of wind.

> Thank goodness civil engineers do not build by trial and error.

Oh but they do. Hundreds - if not thousands - of years of trial and error and catastrophes and deaths.

But the failures get written up, taught, remembered, and applied to future projects.

(And even then not perfectly - witness Mexico City and buildings on a drained lake in an earthquake zone; or Houston's wild building expansion in a hurricane-prone floodplain.)

Software, in contrast, has barely 60 years under its belt and a culture of hiding failures. Give it another 500 years and it'll start to look more like the architecting and building industries we have today.

> code is a more useful blueprint than anything else for programs

This is so clearly false that it can actually be mathematically disproven. To prove that there are better software blueprints than code, I will prove that there exist algorithms/systems for which sufficient formal blueprints exist, yet no code can capture. To prove that, I will need to define the terms more precisely. A "sufficient blueprint" is one that makes it possible to prove properties of interest about your software. "Code" is a formal representation of an algorithm, which always allows efficient mechanical execution, meaning, mechanically executing code is within a polynomial bound of the complexity of the algorithm that code expresses.

Now for the counterexamle: take the specification of the Quicksort algorithm, [1], shortened here as: 1. Pick a pivot. 2. Partition the elements around the pivot. 3. Recursively apply 1 and 2 to the resulting partitions.

I claim that this specification is sufficient. For example, from a direct formal statement of it, you can formally prove that it actually sorts and that it runs in worst-case quadratic time complexity. Yet, I claim that no code can express the above specification; in fact, no code can express any of the three steps. For example, step 1 says "pick a pivot", but it doesn't say which. This is not an omission. It doesn't say which because it doesn't matter -- any choice will do. And yet, code must necessarily say which pivot to pick, and when it does, it is no longer captures the specification, and is no longer a blueprint for all implementations of that specification. Similarly for the other two steps: code must specify which partitions are created of the very many possible and completely fine ones, and in what order the recursion takes place. QED

Languages like TLA+ can formally, i.e., mathematically, capture precisely the specification above and express anything code can. There is no magic here: Efficient execution of code (provably!) cannot be done in the presence of some quantifiers, yet quantifiers increase the expressiveness and "generality" of the language.

[1]: https://en.wikipedia.org/wiki/Quicksort#Algorithm

I feel like this is part of why Brooks says build the first one to throw away, because you're going to.

People seem quite unable to imagine life with a program that doesn't exist. Building a more precise specification would be valuable if people were really able to evaluate the specification as if it were a program, but they don't seem to be able to do that. So we wind up building the whole thing to see if it's right, and it never is, but then we build the next one, and it's much closer.

They are able to do that. But, unlike the physical world, it's really expensive compared to just building a virtual one first. And then really tempting to just use that prototype, or patch it up.

If it was super cheap and easy and almost free to build things in the physical world we'd probably do the same thing and just skip the architecture and wing it.

> Brooks says build the first one to throw away

A great theory, but all too often, companies have a hard time scheduling time for a rewrite, and the hacked-together rough-prototype code gets pushed into production in perpetuity.

And that's because companies have learned that customers usually aren't patient enough to wait for something rewritten. Most customers want something now, even if it's not perfect, rather than wait for something better. So, to survive in a markplace with these types of customers, companies have to shorten time to market...

Well, even that explanation would imply that the company should give you time to at least fix the worst hacks after they've released the first version (which will probably have problems anyway), but that also doesn't seem to happen...

I'm more and more wondering if the real reason isn't that business people have issues with technical terminology. In this case: A prototype is something you throw away before you build the real thing. Yet somehow many non-technical people think its meaning lies more along the lines of "beta".

I've seen other misunderstandings like people describing security-focused static analysis as "pentest". Each time I'm just left scratching my head...

Solution: Be pedantic in your usage of words and correct people around and before you all the time if they use a word wrongly.

Let's say it that way: If that's an effective way for you to deal with this, I envy you for your work environment.

I code for a couple of the other groups in my division as a favor. I still run our entire QA group out of a folder called "mock database" for this exact reason.

I feel like maybe they simplified Lamport's sentiment here, because Lamport's on record as pointing out that often times the minimal specification of code is the code itself, and that's a very big difference from the architectural world.

Me too.

The very few design specs I've seen during my 15+ yrs as a programmer are typically high-level enough to be the mechanical equivalent of "cogs needed here, and springs!". What kind of cogs, e.g. diameter, teeth count, diameter of the center hole (should the hole even be in the center?) and what kind of springs are never mentioned.

If you ask the customer about functionality their response is "we want it to work and we wanted it yesterday!".

So what you have is an intuitive feel of what the customer wants and so you write some code to see how it could come together. Then you show it to the customer to see if this is what they wanted and BAM!!

The sales rep at your company just sold it to five other customers while you were at coffee break waiting for the response of the original customer. It's barely a prototype, not even an alpha. It's five lines of code and a mock up UI. Bug reports start piling in. The sales rep keep pestering you every five minutes at Skype/email/phone/whatnot because the customers refuse to pay before the bugs are fixed and the sales rep is not getting his bonus before the money is in.

The filed bugs are actually just badly concealed feature requests and the software for which you just started getting a nice mental model over starts to grow the most hideous excrescences. Especially as you have to hire a bunch of new developers just to keep up with the piling bug reports. There's no time to train them or even teach them what the software is supposed to do.

  "What do I do? Where do I start?"
  "Look at the bug reports and fix them!"
  "Which should I start with?"
  "The ones labeled 'blocker'"

But they are all labeled 'blocker' because the sales rep has write access to Jira. You tried to prevent that but then the sales rep escalated all the way to the CEO:

  "I need write access so I can fill in info from the customers!"

Sounds perfectly safe and sane. It's not. But you don't have time to think about that because the sales rep keep Skyping you links to Jira asking "Is it done? Is it done?". You uninstall Skype and your email client and shut off your phone just to get some work done. After an hour the sales rep comes by with the CEO: "What are you doing? We need this fixed now!" Then the yelling starts. You quit your job. Get another. It's the same deal.

The software is FOOBAR but just usable enough that the customers put it everywhere. Airplanes start to crash. Cooling systems at power plants start to fail. Nuclear systems for Mutual Assured Destruction start to detect incoming missiles.

THE END

This is brilliantly funny, you made my day. In all seriousness though, go find a new job.

You had me at 'and springs!'

Exactly.

Software is not eating the world, the world is eating software, and the world is starving and just wants to shove it down without even tasting it.

Sometimes the world gets food poisoning for eating raw software.

Software developers are just the cooks trying to tackle this impossible task.

> Sometimes the world gets food poisoning for eating raw software.

This is my new tagline. Thank you.

Sounds like you should have gone to the CEO and got there before the sales guy did....

> Sounds like you should have gone to the CEO and got there before the sales guy did....

He did the right thing: He did quit the job.

No, seriously: If you work at a company where the CEO gives the salesperson that kind of permission (with the described consequences), you really better quit, since it is by definition a dysfunctional company.

"Everything should be built top-down, except the first time" --Alan Perlis

This was the last straw. After many years of seeing Alan Perlis's Oscar-Wilde-style epigrams about programming show up everywhere, I finally looked him up.

It turns out he published all those epigrams together in a single document in 1982: http://pu.inf.uni-tuebingen.de/users/klaeren/epigrams.html

An architect's product is not a building. It is a plan for the building. So the architect is doing what the software engineer is doing, he's iteratively drawing his blueprint.

One problem I see with that quote is that it is presuming that all programmers choose to just dive in at their discretion. It ignores the fact that often programmers are simply told to just dive in and deliver quickly. It's simply a game of mixed attitudes on project management delivers mixed results.

I almost always write the major steps as comments (beginning with 'TODO:') first and then do the implementation once I'm satisfied that I know what a good solution looks like. Over the years I have had the misfortune to discover way too many functions that just were 50% implemented or simply did something that was not at all what they were supposed to do.

I don't doubt that conscientious developers can arrive at good code however they prefer. But there are a lot of people out there who will start with a quick and dirty implementation that they then debug into a more or less functional state. I see a push towards making people stop and take a moment to think about what they are going to do. Test-driven development is another tool to achieve that end.

That's more like the carpenter drawing marks in the wood to tell him where to cut.

I feel like, complexity is a problem when there is more software than one person could write. At this point, you probably need to do some Business Analyst type stuff.

For instance, we are aware of race conditions in code, and maybe we can avoid them by writing purely functional programmes in rust or whatever people do these days.

But what about the race condition of several servers restarting repeatedly and coming back up in a different order? Or a customer registering online when he already sent a form in by post?

The oldschool way of doing this was big UML drawings. This was a compromise because both programmers and business subject matter experts hate drawing them equally.

People don't tend to do this anymore because it is 90s and smells of XML and Java. But what replaced it?

Note that Rust doesn't prevent race conditions. Race conditions are application specific. Rust prevents data races, which has a very specific technical definition: an unsynchronized read and write to the same location in memory.

Programming is making the detailed plans - the building phase the one that is analogue to the brick laying or hammering nails is done by the compiler. Of course this is not an argument against making less detailed, more overall sketches - they are still useful - it is only that that particular argument for it is incorrect.

By the way the article makes another similar mistake: "Instead of writing normal programming code, you created a model of the system’s behavior" and then it talks about how programmers like to code and oppose this new approach - but this model creation is still coding - the code is visual and a little bit more high level. But building this model is still programming.

> But building this model is still programming.

I agree with you, but I don't think this is a mistake in the article. I read it as normal programming code meaning low level procedures as are typically written these days.

More and more I'm writing high level pseudo-code, and planning ahead, before diving in. When I'm done, I basically have a to-do list and a good chance of not painting myself into a corner, which would otherwise happen way too often.

I tend to write code four times following the same principles, the first two are similar, then the client explains his real needs and I have to start again.

So basically all our problems will go away if we return to UML and requirements documentation that's hundreds of pages long and edited for ages before anyone writes any code.

No. Have a look at this video. It's worth it. He's just saying that you should think before you code and that most specifications should be just a few sentences stating what you intend to do. Code as a blueprint is not the same.

https://youtu.be/-4Yp3j_jk8Q

Thats how most Medical Device software is written :)

Fair enough but it's hardly a panacea. I think it likely does work better with something like that where the requirements are truly well understood, though. Most of this "agile" stuff is working around people not being able to tell you what the hell they want.

I was intern at a company that writes medical software and software for medical devices. I tell you: The usual standards are mostly bureaucracy and rather worthless in that they change anything about development practice.

I also had a nice private talk with a person who worked in software development for avionics at some very well-known company. Before I talked to him, I thought that at least there the development practices are better. After that I seriously ask why there are not serious accidents about everyday and how the plane even comes off ground. :-(

TLDR: Dream on... :-(

That sort of development practice has a much closer relation to engineering than current practice does: it's just that it's extremes are on the other end of that one.

The difference is that engineering has to get it right the first time. There is no second try. Most software isn't like that. It gets continually updated and maintained over several years after the initial release.

> So basically all our problems will go away if we return to UML

UML belongs to the class of semiformal models. In other words: It leads people into believing that a lot more things are precisely specified than actually are.

Reply on news.ycombinator.com