Mezzano: Operating system written in Common Lisp | Hacker News Reader

Mezzano: Operating system written in Common Lisp

github.com

224 pointsingve68 comments

Open articleView on HN

Some starting points:

USB mass storage driver: https://github.com/froggey/Mezzano/blob/master/drivers/usb/m...

ext4 file system (read-only): https://github.com/froggey/Mezzano/blob/master/file/ext4.lis...

TCP network stack: https://github.com/froggey/Mezzano/blob/master/net/tcp.lisp

https://github.com/froggey/Mezzano/blob/f0eeb2a3f032098b394e...

I'm not sure what I was expecting, but 110 lines of keyword arguments was a surprise.

Better that 100 lines of non-keyword arguments!

Does anyone find Lisp readable?

I haven't used Lisp professionally, but find the code terribly confusing to read (mostly due to the look of it, I assume), while I can usually follow code of an unknown language reasonably well.

Yes I find Lisp very readable.

You get used to it pretty quickly. Just need a bit of practice.

I used to say that proper indenting helps readability but these days I have departed from the standard indenting and do my own thing.

That said, this guide on formatting lisp is very useful IMO if you want to learn to read lisp code better:

https://link.medium.com/idMRZk93Fdb

I actually wrote it, but now I don’t follow many of its principles :D perhaps because I got a deeper understanding of it. Check it out if and see if it helps

What is the reason why Lisp allows you to use the hyphen for an identifier name?

This overloads the subtraction operator. And it makes the code very confusing to read.

And plus, Lisp also uses the hyphen as the subtraction operator.

It doesn't. Common lisp is a lisp-2, which means that there are 2 namespaces: one for functions and one for variables. (Technically there are quite a bit more, but that's beside the point.)

Which means that you can do this:

  (let ((- 5)) ; bind the '-' variable to 5
    (-         ; call the '-' function
      -        ; on the '-' variable
      17))     ; and the number 17
               ; this evaluates to -12

You can. But you shouldn't :-)

this is also possible...

  CL-USER 5 > (- 1 2 3 4 (third -))
  -10

This is a non-issue for all Lisps, not all of which are Lisp-2.

Your code doesn't seem like a very good idea, even though it is legal.

> Your code doesn't seem like a very good idea

Why not?

I usually use an 'anaphoric if' macro that binds - to the given value. It seems more clearly 'special' than a name like 'it'—which seems like the canonical name—closer to the perl/raku $_.

That's more of an issue for languages that use infix notation for operators. Lisp uses prefix notation, that's why

a-b

is not a problem. As a practitioner, you don't get confused at all when you encounter a variable name that contains a dash, because you don't even read it as a subtraction. Your brain is trained to match only the

    (- a b)

pattern for that.

C programmers on Unix somehow don't get confused by the POSIX shell:

  $ foo-bar()
  > {
  >    echo 42
  > }
  $ foo-bar
  42

or Make:

  OBJ_COMMON := common.o

  OBJ-$(CONFIG_FOO) += foo.o foo-tab.o
  OBJ-$(CONFIG_BAR) += bar.o

  OBJS := $(COMMON_OBJS) $(OBJ-y)

even though foo-bar and OBJ-y look like a C subtraction.

C programmers on Unix came up with this stuff themselves.

in a more traditional sense Lisp does not have operators by default. Operators which have a special syntax (infix ...) and precedence rules. Operators like + and - are written as normal functions -- thus these characters are allowed in function names. Actually more characters incl. whitespace and returns are possible -> one can escape characters in symbols or whole symbols.

There are a bunch of conventions for naming things:

  +foo+     ; convention for naming a constant
  *foo*     ; convention for naming a special/global variable
  foo-bar   ; convention for a compound name

Other programming languages use FooBar, foo_bar or similar, instead of foo-bar.

More naming conventions in Lisp are collected here:

https://www.cliki.net/Naming+conventions

Thanks for the link - very useful

In my non verified view, it’s easier to type with dashes and looks better than camelCase. Also spaces are used as delimiters, so can’t use them.

It’s all preference though. I used to like camelCase when I was learning Java, now I like hyphens. I program both in JavaScript and Lisp and have no issues with both

I may be wrong, but one of the benefits of NOT allowing hyphens in names in languages like JavScript is code can be minified (you can remove spaces and use operators as delimiters, which only works if “-“ only stands for subtraction).

I think. Haven’t verified this, but the thought came to my head the other day.

There is not a great need to minify many languages other than Javascript. The way I see it, the need arose mainly because it took such a long time to agree on a bytecode representation of JS.

I would say that Javascript was not at all designed to be delivered the way that it is. The design characteristics of JS that allow minification, I think are just happy coincidence.

Consider that Javascript early on was marketed as a companion to Java (applets). Java, of course, used a bytecode representation to deliver programs on the Web from square one. If Javascript had been designed for efficient delivery of "compiled" programs, bytecode would have been the obvious choice.

I think my point is, languages are not generally designed to minify well, because minification is the technique of last resort for efficient delivery.

Thanks - this helps expand my knowledge on this topic :)

I think a much bigger consideration would be the utter difficulty of reading something like this:

  a-b - a - b

Even though I personally love hyphenated-identifiers in Lisp, I would not want them in any language that has infix operators, whether it cares about minification or not.

I think it’s getting used to it, for example I would never confuse a-b with a - b, but that’s because I’m used to using hyphenated identifiers.

I can see others (beginners) getting confused, but it’s something they will very quickly get over. For example I didn’t know JavaScript was case sensitive, but it was a very quick adjustment when I learnt this

In some infix syntax, one might then need to escape characters. Example 'infix' for Common Lisp:

  (let ((foo-bar 29))
    #i(   foo\-bar - 3 * foo\-bar    ))

I refer to long symbol names a lot more often than I subtract things. If I could have it in Python or Java I'd be happy, using - is just nicer than _ or camelCase. Using spaces in pipes |like this| is nifty sometimes too. I'd be willing to accept warnings/compiler errors on the seemingly never-in-practice occasion when the compiler notices that all 3 of 'a-b, 'a, and 'b are bound symbols and I need to use a more explicit expression (like the escaped '- in lispm's example, which is how the cmu-infix library solves the problem).

On the other hand, some advice from Chuck Moore (found in Thinking Forth) that I think about sometimes:

"There are diverging programming styles in the Forth community. One uses hyphenated words that express in English what the word is doing. You string these big long words together and you get something that is quite readable.

But I immediately suspect that the programmer didn’t think out the words carefully enough, that the hyphen should be broken and the words defined separately. That isn’t always possible, and it isn’t always advantageous. But I suspect a hyphenated word of mixing two concepts."

That could be English written in ASCII, with the same hyphen serving as a dash:

Example sentence:

"Examples of sado-masochism - such as Java-influenced languages - are rampant in the field of computing."

Quite readable.

> What is the reason why Lisp allows you to use the hyphen for an identifier name?

Lisp has only ~7 special characters: (, ), #, ', ", ` and wite-space. All others are valid as part of an identifier, alone or together. So for example "---" is a valid Lisp identifier, just as much as "aaa".

There is no subtraction operator in lisp. "-" is simply the name of a function in the standard library that implements addition (technically, it might be a special form and not a function, but that's beside the point).

> This overloads the subtraction operator. And it makes the code very confusing to read.

a-b-c in Lisp is unambiguously an operator. Subtraction would be (- c b a) - you can't really confuse these two.

Subjectively, I happen to think that hyphenated-names are much easier to read and type than camelCase or snake_case. I think most of the world agrees as well, since this is one of the more common typographical roles of the hyphen: joining words, such as 'pre-approved' or 'ill-defined'.

Haven't seen anyone mention that "-" is not a function at all. it is a symbol. A symbol which bound to the function which does subtraction.

Users are free to redefine it outright, or to redefine it it multiple ways in different scopes of their application. (Not sure how wise this would be, but not really the point ...)

Not to be offensive, but the question as written seems to imply looking at lisp from preconcieved notions from other languages that probably don't make sense to carry over in understanding lisp.

One thing about Lisp: it's a set of languages within a language. Whether you understand it depends on which of the sublanguages you're reading.

`loop` has its own special conventions. People write macros. And so on.

I recommend giving it a shot solely for the fun of it. It's worth the payoff, even if it takes awhile.

Yes, func(arg) becomes (func arg).

Yes there are lots of parentheses, just like there are lots of ampersand, asterisks, curly and square brackets on C derived languages.

Yes, I personally find it pretty easy to read, even though I've only used it as a hobby.

The only problems I have are when I'm trying to debug something in macro code, where it becomes too hard to intuitively follow the layers of quoting and unquoting to understand what's going on.

I find it very hard to read. The reason is that it almost completely lacks demarcation between different things.

Expressions should be readable in a single flow. Lisp breaks that.

For example:

    (+ (* 5 3) (* 3 2))

This is not easily readable because there are breaks in the "flow".

You need to go like:

* `+` is going to add things together

* oh, there's a nested expression here, pause...

* `` is multiplying 5 and 3 - cool, back to the previous thing.

oh, another nested thing.

* `` is multiplying 3 and 2 - back again...

done... so where were we?

Here's how a readable expression would look like IMO:

    (5 * 3) + (3 * 2)

This reads very naturally:

* 5 times 3 - 15, ok go on...

* nested expression... hm, ok.

* 3 times 2 - 6, let's go back now...

* ah right, 15 + 6 is 21... done.

Alternatively:

    5 3 * 3 2 * +

* take 5 and 3, multiply: 15

* take 3 and 2, multiply: 6

* add them together: 21

Which is what FORTH does. This is almost perfect, but a little hard for humans to follow (though this is exactly what machines want to do) because you still need to keep the stack of values in your head.

I've been studying about this and came up with something in the middle:

    (5 * 3) (3 * 2) +

This has the advantage that you the distance from the values in the stack to the point of reading when going left-to-right is optimal: there is no better way to represent this if your objective is to avoid intermediate values.

It is similar to FORTH, but optimized for humans to read instead of machines:

* `5 * 3` is 15.

* `3 * 2` is 6.

* `+` the values, which are 15 and 6: 21.

The parens are not required if you use some alternative syntax to mark boundaries, for example:

    5 * 3 >> 3 * 2 >> + ;

The `>>` is used to "pipe" the value to the next expression, while `;` is used to terminate the expression. This also has the benefit of showing the reader the direction of the flow.

  (5 * 3) + (3 * 2)

That's not that great, since it uses infix precedence - either explicitly (here) or implicitly. I need to read into it to understand that a sum is computed and I need to then find out the boundaries of the subexpressions. But the brain can be trained for that, just as it can be trained to read Lisp, Erlang, or even APL code.

Nested lists are the base for Lisp syntax. Thus numeric code is just like other ordinary code. Lisp was developed to also do symbolic computations: computing with expressions with symbols.

Thus to read Lisp you need to learn to read syntax based on nested lists and then numeric expressions are just an application of that. Still one is trained to do math with lot's of fancy syntax - fancy syntax you won't find in normal code anyway. For example the typical graphical root notation is mostly not found in programming languages.

If I (as a Lisp programmer) had problems understanding numeric expressions, I'd reformat them:

   (+ (- a b) (* 100 (sqrt y) x) (* 50 (expt x 2)))

   (+ (- a b)
      (* 100 (sqrt y) x)
      (* 50 (expt x 2)))

That's just another way to format a nested list.

The main problem is mapping that back to existing external forms which uses mixed infix/prefix/postfix syntax.

   (a - b) + (100 * y^1/2 * x) + (50 * x^2)

Lisp allows one to compute this stuff:

  CL-USER 18 > '#i( (a - b) + (100 * y^^(1/2) * x) + (50 * x^^2) )
  (+ (- A B) (* 100 (EXPT Y (/ 1 2)) X) (* 50 (EXPT X 2)))

  CL-USER 19 > (let ((*print-right-margin* 30))
                 (pprint *))

  (+ (- A B)
     (* 100 (EXPT Y (/ 1 2)) X)
     (* 50 (EXPT X 2)))

> That's not that great, since it uses infix precedence

This is not true. I was careful to not use operator precedence as I agree it's bad. I said that the syntax I had in mind reads left-to-right, hence the operators also work that way. So `5 * 3`, assuming `` takes 2 parameters, is all you need to know about the operation (you don't need to check what's next, you have no precedence to worry about).

> to read Lisp you need to learn to read syntax based on nested lists

Exactly! That's why it's so hard. Our brains are not good at keeping several levels of nesting. Any code that is deeply nested is inherently hard to read.

> But the brain can be trained for that,

That's something we should avoid to make programming more approachable. People already have a hard time learning other more important things, so let them use their mathematical intuition for this.

   (a - b) + (100 * y^1/2 * x) + (50 * x^2)

You seem to not have fully understood my point. This is not what I am proposing as being better than Lisp.
This is what this expression looks like in my proposed notation:
a - b >> 100 * x * sqrt y >> x ** 2 >> 50 * >> + >> + ;
There's zero nesting, but you still have to keep a few intermediate values (that's inherent to the calculation and this approach keeps that to the optimal minimum).
I thought it was obvious, but let me point it out explicitly: there is no operator precedence in this notation.

I believe this is fundamentally better than Lisp and C-like languages.

  a - b >> 100 * x * sqrt y >> x ** 2 >> 50 * >> + >> + ;

That's a mix of infix and postfix. I don't think writing code like that will be very readable for most people. Humans are much more trained for prefix and infix notations, than for postfix stack operations. We don't see widespread usage of postfix notations in programming languages besides languages like Forth, Postscript or Factor.

The Lisp notation has a different reason why it has been invented: it is based on a data structure around nested lists of symbols. That's a very simple and powerful model. code is written as data. Lisp programmers find the code as data idea attractive. Without that, Lisp would have kept the original idea of a conventional syntax.

> Humans are much more trained for prefix and infix notations

Non-programmers are not trained for prefix notation. Only Lisp programmers use prefix notation for mathematics, so I don't believe your assertion has much evidence to show for it.

> The Lisp notation has a different reason why it has been invented

Yes, and that reason was not readability. That's what I am trying to show.

> code is written as data

Which is great fun, but most programmers don't need or want that.

; prefix notation for mathematics

Mathematicians use that: cos (sin x)

; Yes, and that reason was not readability. That's what I am trying to show.

Not convincing so far. It turned out that the notation was the most practical (writing, reading, manipulating) for its purpose, otherwise the original syntax would have been used.

> Which is great fun, but most programmers don't need or want that.

Let's bring the fun back into programming.

I use a rainbow parentheses addon for my editor. Makes stuff readable even when on a single line.

Readability and maintainability are pretty much the only reasons why I wouldn't recommend Lisp for everything. Other than that, Common Lisp has had practically all features you could desire for decades already.

It's not a problem with the syntax, though. The problem is that Lisp is so powerful that programmers can and will create very high level abstractions, the language and conventions encourage it, but a side effect of this is that every part of a program has its own DSL. If in addition there is not good enough documentation or a programmer has a knack for obscure elegance over KISS, then this becomes problematic.

In a nutshell, it's perfectly possible to write clean and easy to read Lisp but not many people do it.

Edit: I forgot to say that the code from Mezzano looks like a positive example of readable and clear-cut Lisp code.

I completely agree, and it’s also a problem that other immensely powerful languages like Ruby and C++ have but Lisp has it far worse since A) the more arcane abstraction features are more immediately accessible from day one of writing Lisp code and B) the culture of Lisp code is far more obsessed with elegance and with twee little tricks than with readability or simplicity. It’s not a necessary issue that Lisp dialects have though: Common Lisp seems to be better as far as culture goes, and Racket has both a more simplicity-focussed code culture and the tools to make the arcane and elegant abstractions easier to understand.

Yup.

Time to refer to this image again I guess: https://www.thejach.com/imgs/lisp_parens.png (Though that's technically Scheme -- I find Common Lisp more readable and encouraging of smaller functions and modularity. The clean code culture of tiny functions and unit tested everything is not there, though, and a lot of stuff is done by individuals so the benefits of code review aren't often captured.)

After spending about a year toying with LISPs I find lisp prefix notation much more readable, intuitive and useful.

Think about adding a bunch of numbers using infix

a+b+c+d

In prefix it’d look like

(+ a b c d)

You could as many terms in that list.

And as far as the - in indentifiers, it’s called kebab case and I find it very readable and natural to type as well. Also since identifiers are a lot more flexible than other languages you could have predicates such as empty? Rather than isEmpty and so on. Very very friendly and useful. Not trying to convince you or anything but if you give it a chance it’ll click for you.

Well, I can understand (+ a b c d)

Its the next level where it gets hard.

What is that, macros? Hygienic macros? Higher order functions? Recursion? I find that having such a simple syntax everything is grokkable much easier than in other languages. Having said that, I prefer scheme and racket which are much more simplified than common lisp and don’t contain so many warts.

To someone who isn't immersed in the Lisp world, Lisp syntax often looks ((((something (like) this) and) seems)) utterly impenetrable compared to a more 'conventional' syntax. It's the reason for the snarky backronym, Lost In a Sea of Parentheses.

Forth, another language with a spartan approach to syntax, gets similar criticism.

Kind of like saying C often(looks{something{like->this;}});

Overly dense C can be cryptic, sure, but Lisp relies heavily on parentheses. At least in C and C++, there are several types of brackets at work (parentheses, square brackets, braces, and in C++, angle brackets and even double square brackets [0]). This helps the eye get a grasp of things.

At least, things seem that way to me, as someone who knows C and C++ reasonably well but has only dabbled in Lisp.

[0] https://en.cppreference.com/w/cpp/language/attributes

Well that is kind of the point -- you know C and C++ reasonably well and are comfortable with the syntax, so of course you think it is easy to read. If you were as familiar with Lisp you would not find it hard to read either (I should know, since I am comfortable with both C++ and Common Lisp).

That said, if you want a real C++ readability challenge, here you go:

  int add_one(int x) { return x+1; }

Easy-to-miss undefined behavior is a much greater readability problem than the specific syntax of a language in my opinion.

I agree that UB is a serious issue in languages like C and C++.

I thought you were going in another direction with that example, the Most Vexing Parse. https://en.wikipedia.org/wiki/Most_vexing_parse

Lisp always uses prefix operator symbols. something like { ... } would be (progn ...) . Thus when one reads Lisp, one doesn't look that much for character syntax, but for operator names and list structure layout.

Parentheses are there, because s-expression syntax is not only used for syntax for nested lists, but also for representing code as s-expressions. C / C++ does not have that feature.

These s-expressions in Lisp are data. This means one can also write programs which create programs as data - not as text, but as list structures.

> when one reads Lisp, one doesn't look that much for character syntax, but for operator names and list structure layout.

Right, but this boils down to just getting used to Lisp.

> also for representing code as s-expressions. C / C++ does not have that feature.

Sure, I know enough Lisp to understand that, but the point about syntax stands.

Ultimately I think it's just a matter of acclimatisation. C++, Lisp, and Forth, are different enough that if you only know one or two of the languages, you'll likely find the third to be an unreadable mess.

What you're looking at though in lisp is an unambiguous syntax tree.

It's what helps you find your way around in the forest, especially with editor support that can automagically apply operations to discrete chunks of your code.

That can include doing things like re-defining a function or value on the fly, which is one of the lovely things you can do in many lisps.

> What you're looking at though in lisp is an unambiguous syntax tree.

...

> That can include doing things like re-defining a function or value on the fly, which is one of the lovely things you can do in many lisps.

Sure, but like I said to lispm, I know enough Lisp to understand both of these points. Respectfully, neither addresses my point about syntax.

I find Lisp to be at least as readable as anything else, and much more readable than C++. The syntax is different from other languages but that is not a problem once you are comfortable with Lisp syntax. Lisp syntax and semantics almost always mean that there is less need to work around annoying and brittle aspects of the language, and so the code you are reading tends to be more relevant.

Readability in any language basically comes down to whether the programmer formatted his code in a non-malicious way (which is the case 99% of the time) and used variable names you are familiar with. I'd be more worried about the meta constructs in Lisp (and whether they are sound). To be able to reason semantically about a program is something that is made easier by static typing and static name resolution. Instead of constantly worrying that X is not defined where you think it is (because some dynamic binding system has yet another edge case you forgot to think about or discover), you just always know where it's defined.

Yes. Lisp is easy to read.

Most recent release, Demo 5, from July 2020: https://github.com/froggey/Mezzano/tree/demo5

Previous discussions from a few years ago:

https://news.ycombinator.com/item?id=23950558

https://news.ycombinator.com/item?id=20210227

https://news.ycombinator.com/item?id=14072121

https://news.ycombinator.com/item?id=12358177

https://news.ycombinator.com/item?id=8955140

I looked through your links, and I saw that it was renamed from Mezzanine, so there's at least one more discussion:

https://news.ycombinator.com/item?id=8943709

It looks like it isn't capable of building itself as yet: https://github.com/froggey/MBuild

Hopefully one day it reaches that point. Ability to build one's self, and even to host one's own development, is an important sign of completeness in an operating system.

I found this comment funny. From the colour.lisp code.

https://github.com/froggey/Mezzano/blob/master/gui/colour.li...

         ;; Mangle & unmangle the vector ordering so the colour matrix
         ;; layout makes more sense. This could be fixed up by rearranging
         ;; the matrix layout, but that's a bit beyond me.
         (new-rgb (matrix4-vector4-multiply
                   matrix
                   (mezzano.simd:shufps original-rgb original-rgb
                                        #4r0321)))

Just ran it on a VirtualBox.

This is beyond amazing.

How long have you been working on this for?

How do you start writing an OS in a language other than C?

Exactly the way you start writing an OS in C. You implement just enough of the language runtime to boot to a hello world, and then iterate from there. C has a runtime (stdin, stdout, stderr, a heap, a stack), it's just so minimal that people don't think about it much.

[edit]

I should point out that it's typical for a common-lisp implementation to have builtin tools for manipulating pointers and generating arbitrary instructions (since both of those are needed to implement the compiler anyways, and lisp compilers are typically self-hosting).

To add to this, there's an OS written in Go in similar way

https://pdos.csail.mit.edu/projects/biscuit.html

You write an ahead of time compiler to machine code for your language. You write a program that is then compiled to native code. You do a bit of assembler glue to set up an execution environment that your language runtime needs. You link them together, do a bit more glue to turn the result into a bootable disk image. Done. You start out in kernel mode and will have to do a lot of work to bring online a conventional OS-like environment, but from that point on there's nothing really C-like required.

This needs a native Raspberry Pi port, it would be an amazing workstation.

Agreed.

https://github.com/froggey/Mezzano/issues/46

What a neat project. I think I’d enjoy contributing.

Impressively small and clean code base for that functionality.

Reply on news.ycombinator.com