The TTY demystified (2008) | Hacker News Reader

The TTY demystified (2008)

linusakesson.net

168 pointstim_sw40 comments

Things like this, with quirks due to decades and decades of history, always make me wonder what things would be like if we started over, with a clean slate. In so many places, when you start digging, you find something that references the way mainframes worked in the sixties or teletypes way before that etc. I mean, it's probably not practical, and it would probably be killed by committee or reimplemented in JavaScript anyway, but you can't help but wonder...

There are a few groups working on reimagining computing from first principals. VPRI is one such group:

http://vpri.org

The most interesting work I know of from VPRI is the STEPS project, which aims to design an OS that fits in less than 20000 lines of code:

http://www.vpri.org/pdf/tr2011004_steps11.pdf

Interesting stuff, thanks for the link!

You're welcome klum.

You may also be interested in unikernel operating systems. The designs vary, but the three points below describe the general idea:

* Existing virtualisation technologies (KVM, Xen, etc...) used to manage hardware.

* OS images run on top of this virtualised base.

* OS images consist of a single software application + a minimal kernel that is customised for the needs of the application.

Unikernel-based software has many potential advantages over traditional OS designs, including increased performance (especially compared to other VM approaches), reduced size, better security (though hacks like Blue Pill would probably still apply). The most active unikernel projects are almost all based on functional languages, so you get the benefits associated with functional development too.

The Xen Project Wiki has a decent introduction to Unikernels:

http://wiki.xenproject.org/wiki/Unikernels

If you wanted an easy way to get started, this guide on the MirageOS website walks you through setting up a simple web server that can be hosted on Xen:

https://mirage.io/blog/mirage-seal

For historical reasons, UNIX and Linux handle the line discipline and echoing in the kernel. But really, that should be done in a user-space program. Something like xterm, but without X. The kernel should just handle the serial port hardware.

Is serial port login even enabled by default any more on Linux?

I agree it should be done in user space, but how? Surely requiring every program to use something like GNU readline is a bad idea? I vaguely remember reading somewhere that Plan 9 does this well. Does anyone know how, or know another OS that does it well? I also vaguely remember one of my textbooks mentioning something called STREAMS from System V. Anyone know anything about it?

Hurd has a console server.

The line discipline analogue isn't in the console daemon. It's in the term daemon.

* http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/hu...

You are the first person I've seen to ever include the rightmost dot in the domain name in a comment on the internet!

> I agree it should be done in user space, but how? Surely requiring every program to use something like GNU readline is a bad idea?

Not full readline, but libtinfo, or a similar library that reads a terminfo database. And the standard C library could handle this by default for "cooked" input devices.

I find this idea not very attractive. That means any program that reads from stdin has a dependency on some random database. Even cat, or grep. (You could argue that grep needs this anyway to color things, but still.)

Not every program that reads from stdin, just every program that wants stdin from a "cooked" TTY. Many programs can just perform raw reads.

In QNX, the serial console, if present, is managed by a driver in user space, loaded with the kernel as part of the boot image. That's how all initial drivers in QNX start up. For embedded systems, the serial console driver might not be loaded, or could be a special-purpose driver for the hardware. The user mode driver is used for any messages during boot; there's no special startup-time kernel driver.

The boot image may be in ROM, and on small embedded devices, it usually is. For QNX, disks and file systems are optional. If disk support is desired, some disk drivers and an init program are included in the boot image, and the later phases of startup are somewhat UNIX-like.

It's possible to build a QNX boot image so that networking comes up before the file system, which is useful for headless embedded systems. It's also possible to build a classic system for x86-type BIOS-controlled boot from disk, and run QNX on the desktop, but that's not done much any more.

How realistic is this on a monolithic kernel like Linux, though? On QNX all drivers - not just the serial ports - are in user-space. QNX has a microkernel.

User-space drivers on Linux are always a special case - like FUSE. Someone always has to write bespoke infrastructure to make it happen.

Linux has, over the years, acquired many of the features of QNX - interprocess communication, real-time scheduling, user mode drivers, user mode file systems, etc. But all the old in-kernel stuff remains, resulting in a huge, bloated kernel with a huge attack surface.

(If only QNX, the company, weren't so screwed up. They were sold twice, once to Harmon (car stereos) and once to Blackberry. They went open source, closed source, open source, and closed source again. Most developers got fed up and went to other platforms. QNX powers most of Boston Dynamics's robots; when you need coordinated control of all those actuators at 1KHz, you need something at least as good as QNX.)

Wrong side of the tty in my opinion. Everything that is done by tty should be done by the terminal, except generating signals from keyboard input (because that needs to happen on the same machine as the controlling process).

Only reason its in the kernel is that you couldn't patch a VT-100. Virtual terminals are a lot easier to extend than a web browser, and yet they haven't seen a widely adopted extension in more than 20 years.

Working on it. :-)

> Surely requiring every program to use something like GNU readline is a bad idea?

Moving the line discipline out of kernel mode into application mode does not necessitate that every program have an individual copy of the code. As already pointed out, in operating systems that do this the line discipline analogue is encapsulated within a daemon that presents the terminal character devices (or analogues thereto) to the rest of the system. The programs using the terminal think that they are talking to a terminal device in the ordinary (usually, but not necessarily) POSIX-General-Terminal-Interface-like way. As mentioned, in the Hurd this daemon is the /hurd/term daemon. In Windows NT it is the CSRSS daemon, or a CONHOST.

> Surely requiring every program to use something like GNU readline is a bad idea?

That's basically how it works now. Kernel's line discipline is awful and nobody wants to use it so they build with readline or something equivalent.

You'll notice a program is using the line discipline when the arrow keys print weird characters instead of working cor rectly, rlwrap has been invented for those programs.

The clean/logical/plan9-style solution is having the terminal implement line editing. But that alone would preclude autocompletion and history search from working.

> Is serial port login even enabled by default any more on Linux?

That's a distribution choice, but often not. Last I checked on a Debian system, it was just a matter of uncommenting the getty on /dev/ttyS0 in /etc/inittab. Dunno how it's done with systemd nowadays.

  systemctl enable serial-getty@ttyS0.service
  systemctl start serial-getty@ttyS0.service

Alternatively, add to kernel command line:

  console=ttyS0

kmscon is userspace replacement for linux kernel console.

That's a graphical console, not a serial console.

You missed out "a". kmscon is a replacement.

* http://unix.stackexchange.com/a/177209/5132

As you can see, it followed quite a lot of others.

And, as mentioned, it's not what's under discussion here. It (like the others) uses pseudo-terminal devices, with the line discipline code in the kernel.

> Is serial port login even enabled by default any more on Linux?

Yes. systemd even provides it automatically if the kernel's console is configured to be a serial port terminal.

* http://askubuntu.com/a/621209/43344

* http://unix.stackexchange.com/a/194218/5132

In nosh, one simply enables/starts a serial GeTTY service such as mgetty@ttyS1 . There's an example of what such a run script would look like in the "Terminals" chapter of the nosh Guide. It's 2 lines long.

* http://homepage.ntlworld.com./jonathan.deboynepollard/Softwa...

What are the cylindrical metal things on the wall panel in the "1940's teletypes" picture? Is it capacitors?

I see those often in similar old photos, but nothing in modern electronics looks like that afaik so I wonder :)

I have extremely limited knowledge of this subject, but I think they could be transformer coils. For long haul lines one would want to have higher voltage to keep resistance loss at bay.

They're probably thermionic tubes, a.k.a. valves. You want them accessible so you can replace them, but they're also fragile so you put cases over them so you don't bump them.

Here a discussion of three months ago:

https://news.ycombinator.com/item?id=10064657

It's come up 12 times now apparently, I wonder if that's the highest number of HN submissions for a single article:

https://hn.algolia.com/?query=The%20TTY%20demystified&sort=b...

If you're feeling nerdy and missed it you should check out Linus Åkessons other stuff, like his organ and his sendmail turtle.

LFT's "bitbuf" sequencer[1] is one of the better examples I've seen of the concept "leaving money on the table". He released the demo video in 2011 and people are still trying to buy it in the comments.

I would pay good money for just the his code that transposing/modulating live MIDI loop sequences (e.g. from the bass line).

[1] http://www.linusakesson.net/bitbuf/

Really cool.

Why is it called 8-bit music?

The bitbuf is primarily an ATmega88, an 8-bit microcontroller (see his earlier project, the Chipophone[1]). He is also alluding to the 8-bit "chiptune" style of music from the C64/NES era.

[1] http://www.linusakesson.net/chipophone/index.php

Only an American could write this comment. I know this is HN, but life isn't actually an entrepreneurship contest.

This was a very insightful read.

Great article. However I don't under stand how it has managed to be posted 12 times: https://hn.algolia.com/?query=%09The%20TTY%20demystified&sor...

Because:

> Great article.

So are those the rules now? If we consider some thing to be worthy we can post it again and again and again?

Down votes? Anyone care to elaborate?

Seriously, I don't understand. If I find an article that I find interesting I can just post it multiple times under different URL's?

Is that now acceptable on HN?

The official stance seems to be: [0]

> Are reposts ok?

> If a story has had significant attention in the last year or so, we kill reposts as duplicates. If not, a small number of reposts is ok.

It was posted 3 months ago with 31 comments though so looks like that doesn't apply. It's possible that mods didn't see the post (due to the different URL) or that the rules have been relaxed.

As for the downvotes, people in general don't like remarks about reposts because they're off topic and always come off as complaining. It's more likely that most users are seeing something for the first time or if they have already seen it, might not make mind because they like the content.

[0]: https://news.ycombinator.com/newsfaq.html

Reply on news.ycombinator.com