How would you rebuild software from the ground up? | Hacker News Reader

How would you rebuild software from the ground up?

23 pointsregan13 comments

I often see people saying things like: "If you want a secure OS, you'd need to be thinking about security from the beginning." "The HTML/CSS/JS stack is a mess, we need to rebuild it from the ground up."

And Alan Kay certainly seems to want to start everything over.

Are there visions for an entirely reimagined OS, web, programming, etc, that are interesting to you?

If you had infinite time and manpower, how would you do it?

What's intractable today that could be easy, if software were fundamentally redesigned?

However unlikely, do you think there's a plausible path to success for any floor-to-ceiling change like this?

View on HN

Visions which completely reimagine things from the ground up tend to presume a much more spatial experience. Lots of VR, lots of AR. I'd tend to agree, except I think that we're going to see things like brain implants a lot sooner than we think. Certainly before our entire stack can be rebuilt from the ground up. So in a way, I almost view it as futile.

That said, I do have some vague notions on some features a ground-up rebuild should have. Namely:

1. Provably secure systems. I'm talking ironclad. Doing for security at all layers of the stack what Rust did for memory safety, and then some. I want to live in a future where physical access becomes a thing again, so we have more cyperpunk intrigue.

2. Simplicity. Composing software should be like gluing LEGOs together. We have some slick stuff now, but it's not there yet.

3. Visual augments to code. Pure visual programming exists, and it kind of sucks. Whether that's just because we're doing it wrong, I'm not sure.

Rust for example would be a hell of a lot easier to learn if I didn't have to maintain a mental model of what everything is doing at all times, and reconciling that against its vast syntax—half of which is inferred by the compiler—which saves work, but makes the learning curve even harder in the process.

Staring at text is just so banal. I'd love to code in something that has a visual representation for every line of code I write, and see that visual representation come alive. Event architectures in particular would benefit from this.

I think if we're reinventing the entire stack from the ground up, having to not tab out and Google things every other minute would also be nice. Somehow in-context documentation hinting has become shittier over time, not better (in my opinion).

I've a separate vision for a completely reimagined web which I won't be sharing (doesn't everyone?), but it's worth noting that it builds atop the existing stack quite heavily. It's simply not worth it to try and rebuild the entire stack, even in that case.

I think the only thing that could possibly justify rebuilding software from the ground up is if we first rebuilt hardware from the ground up. A radically different kind of hardware that demanded a radically different kind of software.

In order to completely reimagine everything, you have to start at hardware, environments, and operating systems. For the past couple of decades we've been trapped in a more or less "one size fits all" when it comes to all of these. Computer architecture got more homogenized, as did operating systems.

But the web becoming a dominant, near universal platform gives us the ability to experiment with all of these. Just not in the way you'd imagine.

We have the ability to create unique, experimental personal computing models now that will be immediately useful if they fulfill one requirement: they have a functioning, standards compliant web browser (with TCP/IP and all that's needed). If you only have this requirement, then there's a lot of freedom.

Imagine a computer designed specifically for the "operating system" that hosts it (kind of like we used to have back in the days of diversity). I put that in quotes because it could be something LISP or Smalltalk-like, where the language is itself the "operating system." So long as such a thing came with a web browser (and I know it's a big task), it would be immediately useful and usable for most people. That means potential for wide adoption.

A platform like this could then give rise to the "after-web". If the system was an object/actor one, people who had these platforms would inevitably decompose the objects/actors that comprise the built in web browser, pulling them out and giving them additional functionality within the confines of their own systems. Eventually, people would network only these objects with each other. Some standard might emerge from this, and we could have a true network of interacting objects.

Of course, these computers don't have to have text-based languages or anything else. They just have to have a web browser somewhere inside of them. That's what I see as the real promise of the web: it's now universal presence and adoption is the gateway towards the thing that can replace it. But in order to do so we need to re-think personal computing in total.

Great points!

1) To me, security goes hand in hand with your point on mental models. It's hard enough to imagine how a bad actor might abuse a simple system you know well. Scale that up to a complicated system that no one understands and you've got a real problem.

2) Absolutely. For one, I'd want UI separated from data and logic such that you can easily tie programs together Unix-style.

Today, programs have a hard time interfacing with each other. And portability is nasty too. Maybe we could make the _ideas_ behind a program portable?

People quote “Programs must be written for people to read, and only incidentally for machines to execute.” But has this ever really been true? Take any function in your codebase and imagine you were asked to explain it to a friend. Does what's written out in code resemble even remotely what you'd say to them, draw out, or demonstrate? If the representation were good enough, you could just lift out the key idea and add it to your program.

3) I've used LabView a little, and it does kind of suck. Probably because they focused on making the logic itself be visual. When I think about merge sort, I "see" the recursion and merges, with chunks of the list flying around as the stages unfold. I don't see a flow chart or a circuit diagram. I'd bet most people do something similar.

Really, code is a pretty reasonable way to write out program logic. What we're missing is that visual intuition behind it, that mental model we force people to build up imperfectly.

Even if the technology is possible and useful, I personally don’t think we will ever see brain implants used outside of medical applications. The surgery to install a brain implant is dangerous and expensive. There are also religious and social stigmas to overcome where brain implants are seen as something dystopian. Or what about the hipacratic oath of do no harm? It’s hard to justify the convenience of implanted computers when the potential side effects include death. IMHO Brain implants will stay in the realm of science fiction along with flying cars.

Models, Rules, and Schemes all the way down.

Rules are applied to a Model to make more rules, rules are again transformed by rules in stages to compile to targets. Declarative schemes arbitrate between rules, select execution modes such as forward or backwards chaining, hybrid execution, memorization, etc.

Much like the vision in

https://www.amazon.com/Software-Factories-Assembling-Applica...

but more aimed at "getting non-professional programmers to be able to do more easily" and less aimed at "helping a team create software product lines".

Roughly this is the direction that OMG is going in.

> How would you rebuild software from the ground up?

> [ ... ]

> However unlikely, do you think there's a plausible path to success for any floor-to-ceiling change like this?

I wouldn't and no, I don't think there is. Rewrites from scratch most of the time fail miserably, a prominent example being Netscape, who arguably bankrupted the whole company by wanting to start from scratch.

The only reasonable path in my opinion in most cases is the incremental one. This can mean to occasionally extract and rewrite modules but never the whole application or - perish the thought - an entire end-user-facing OS.

Want a secure OS? Linux, particularly with SELinux is reasonably secure. macOS is, too (for most purposes). Both are increments of UNIX. If you want even more security virtualisation could be a useful approach.

HTML / CSS / JS is a mess? Only to people who expect complex, distributed web apps in the year 2017 to behave like single-user, non-networked Win95 applications. There have been massive incremental improvements on the web platform in recent years.

I know this is going to sound more trite than I want, but I can think of no better way to say it.

I'd start by hiring the best of the best. I'd start with paying for quality research. I'd start by giving incentive.

Why? First, while I have opinions, they are unqualified. Second, I am biased because I've seen this work but in smaller portions.

I know, that's pretty generic but it's really where I'd start. I'd like to see security and privacy more easily prioritized. I'd like to see more open standards. And, to be fair, those are pretty generic. I'm pretty sure you don't want a novella with barely literate opinions.

I think the current state of the creating software can certainly be complex but it is so mostly out of necessity and not necessarily bad design. It can also be very simple for simple things and that shouldn't be overlooked or dismissed when we look at the state of the art.

I also think that to conclude "The HTML/CSS/JS stack is a mess" one is really complaining that they want to do something it's not designed to do or doesn't have some specific functionalities they want or need, which is not at all the same as being a "mess".

"Are there visions for an entirely reimagined OS, web, programming, etc, that are interesting to you?"

Well, for me, yes. But we have to look at it with a long term perspective to see it. My first web app (an invoicing app) was made with perl using CGI.pm and HTML. Later I added CSS, then a bit of Javascript to manipulate the UI with Prototype.js and it's fair to say that for me, Prototype.js allowed me to reimagine how to build the app. It allowed me to take multiple forms and "hide/show" them instead of sending user data to the server, saving it, and sending another form for them to continue on with the process of making an invoice. That process was reimagined again when I saw how AJAX worked. And again when I first learned about "Document Databases".

I didn't like the way SQL stored data for creating invoices so I never used it on my app. I used CGI.pm's "Save" feature to store data in a "name=value" format. When "NoSQL" databases, or "Document Databases", came out that were designed specifically for web apps I was pretty amazed because I thought I was the only one who thought SQL was a really crappy way to store document data and the few times I'd mentioned that on the perl mailing lists I got torched by most who responded.

Currently, I think it's probably fair to say that React.js "reimagines" how we approach building web apps.

Finally, when it comes down to the bit level processing going on in an OS I don't have a frigging clue and I know that, so no, I don't know of any better way. I am still in awe and constantly amazed and don't expect to ever not be.

A couple of people have mentioned (or hinted at) formal verification. The problem is, formal verification produces working software much more slowly than other techniques. That plus "start over from scratch" is a bad combination - we wouldn't have the software functionality that we take for granted for a long time.

The original question asks as if we had infinite time and manpower, but we never do...

Have you seen Squeak? http://squeak.org/ You can inspect and modify any code running the system. Wondering what the compiler is really doing? It's right there for you to play with.

Most web devs don't know how a browser works and it makes the web painful and janky. Nobody knows what's going on under the hood, so nobody really knows what their code does, either.

The presentation Building a Better Web Browser[0] by James Mickens is worth watching.

[0] https://www.youtube.com/watch?v=1uflg7LDmzI

Isn't the HTML/CSS/JS stack the only one that is

truly cross platform

with a sandbox baked in

that has a runtime on every tablet/phone/PC on earth?

Software is a means to an end. We can't discuss software without understanding what we need it for.

The ultimate goal is omniscience and omnipotence. If you had that, you wouldn't need software. Software should serve to eliminate the delta between "I want pizza" and "I have pizza".

Humans have physical limitations. They can't be everywhere at once. There's a limit to amount of information they can learn and process. They can't focus on multiple things at once. They need to offload most of these responsibilities to others (whether humans or machines).

Omniscience is the first thing we should tackle. We must capture, process, organize and store the world's knowledge. We can't continue having each company/service/app store knowledge in their own format on their own servers. We need to distribute knowledge in a standard format, that can be consumed and created by anyone. I believe the semantic web was on the right track, and we should push these ideas forward.

Once there's only one knowledge repository, it's trivial to program IoT devices to broadcast all their sensor data in that format.

Omnipotence starts with the formulation of intent. First, we observe the world for problems. Once we identify a problem, we pick an alternative reality that doesn't have that problem. Ultimately, intent is the communication of a future state of reality (ideality), and is represented and stored exactly like the knowledge mentioned previously.

At this point, we have a knowledge base full of data about the real and the ideal. The last step is to bridge that gap, and make the real become the ideal. For that, we need smart contracts, the union of multiple ideals. These contracts can involve other humans, or be automated by involving robots. Resources, whether humans or machines, are defined by the set of all futures they can make happen. For example, a Uber driver is defined by its ability to take you from a continuous set of location A to a continuous set of location B.

Before we rebuild the entire software stack from the ground up, we need to figure out how knowledge will be represented, how smart contracts will work, and what the interface will be like.

The solution will likely implement ideas from:

- AI

- ML

- AR

- VR

- Semantic web

- Datalog/Prolog/Mercury

- Idris/Agda/Coq

- Eve

- lojban

- GTD

- Akinator

- Tinder

- Google Wave

- Google Lens

- Google Inbox

- Google Freebase

- webOS "Just type"

- Apple Watch Time Travel

- Pebble Timeline OS

- Amazon Firefly

- Mechanical turk

- Smart contracts

- Google/Siri/Alexa/Cortana/Bixby

- Messaging bots

- IPFS

- Ethereum

- Bitcoin

- Blockchain

- Edge computing

- WeChat

- Urbit

- Naked Objects

- Squeak/Smalltalk

- 5th generation computing

- Cycorp Cyc

- Homotopy Type Theory

- Xanadu

- React/Redux/Relay/GraphQL

- OpenID

Reply on news.ycombinator.com