Simple Unix Chat
the-dam.org
the-dam.org
http://catb.org/~esr/writings/unix-koans/ten-thousand.html
Master Foo once said to a visiting programmer: “There is more Unix-nature in one line of shell script than there is in ten thousand lines of C.”
Which makes me wonder: Just how much of UI development had software engineering had consumed and automated since 1980s, or is it still in the realm of pure human art?
But then more difficult questions arise: - Is the price of slack worth it, when the alternative is having more educated users that can do very basic command line calls ? - Same question, but taking into account that Slack captures your data and won't give it back to you ? How expensive is it to leave Slack ?
Sure, suc sucks, but we probably could converge on something less bloated and proprietary than Slack ?
I'm not sure that it would be difficult to wrap that up in a web interface running on a single server that does nothing but execute those command-line calls - auth, data and everything but session would be managed by the web server.
You could perhaps even make a local GUI application that spawns ssh once and reads+writes to it, and have your 90% slack functionality done in a weekend.
Wait, there has been a leak from the company so we need some security staff on it and maybe mandated TFA.
And it's nice to have a way to call people and not only chat.
I wonder if there is already an app that does that, even if it's bloated it would be more convenient...
Why? What's wrong with Qt? or any of the other x-platform UI toolkits I already use to target Windows, Linux and Mac?
(Your other points are irrelevant. I never claimed that suc will match slack 100%, feature for feature).
I did not know that. Can confirm that it works in FF on Linux.
But I'd also wager we can go quite far with a single cheap VPS. A few thousands user does not seem impossible at all.
The core value proposition of Slack is next-level beyond that: it adds a visually appealing, highly intuitive UI/UX that almost anyone can sit down and immediately use, without any studying or setup or manuals or training. No technical skills are required. You don't need to understand anything beyond "type username and password"; the rest is self-explanatory.
And yet, there were dozens of other applications doing exactly that (Slack is a list of channels on the left, and the text of the channel in the center).
That said, I'd be really interested in what tools like Slack would look like if they aggressively pursued simplicity ahead of new features. Capitalism as a whole doesn't seem to encourage this kind of development, so there aren't many examples in the wild of enterprise software that is as simple as it could reasonably be. Nonetheless, it's an interesting thought experiment.
The point was that most of these things become less relevant when the surface area of the codebase gets smaller. Individual utilities are generally easier to understand, test and debug as separate units, logging is already baked into the system (syslog, anyone?), and why would you need analytics?
Furthermore, debugging may be "easier" insofar as the units are discrete (though any good modular architecture will have the same advantage), but what do you do when you find a bug? If you're running a business, you can't be in a place where you're just submitting it upstream and telling your customers that every single bug is waiting for a vendor fix. So now you have to take ownership of maintaining and repairing each and every UNIX utility you use. In what world is that easier than having 500k lines of Go in a consistent company style?
Joel Spolsky's advice is pertinent [0]:
> If it’s a core business function — do it yourself, no matter what.
https://www.joelonsoftware.com/2001/10/14/in-defense-of-not-...
Screen sharing and video calls can be handled by other apps like Zoom, Google Meet etc.
Homegrown solutions require varying levels of support. Something like Slack is predictable and that makes it attractive, from a business perspective.
https://thegongshow.tumblr.com/post/345941486/the-spawn-of-c...
ftp/rsync => Dropbox
suc => Slack
...It goes over a utility called "suc" (Simple Unix Chat) that implements server functionalities from Slack, Discord, etc. using a very small codebase.
The novel part is it leverages existing unix tools and methodology instead of re-inventing them.
- Auth is handled by SSH.
- Channels are just a file and admin/mod controls are handled by user groups and file permissions.
- Support for rich text, file uploads, etc. aren't a concern as you just write whatever data you want to the channels and let the client interpret it.
- Bots are very easy as you just pipe to/from the channel files.
You're not going to be making a perfect slack clone with the 5 lines that write messages to the channels from the article, but I'm pretty impressed how far you can with really simple unix tools.
“Here’s how to make $2 million dollars over night”
_reads article_: “have $4 million in the bank…”
That pretty much always explains the “X in unfathomably few lines of code.” No faulting the other, he was very upfront about this.
Existing chat platforms don't "re-invent" Unix tools - they're simply not suitable for building on top of. There's a reason that approximately 0 large production systems are written in shell script using the "everything is a file" paradigm - because it's not acceptable for any non-trivial system that needs to see real-world use.
Not really Slack's fault as far as I can tell.
Almost all modern software is bloated to hell. The idea that one would need 1.7 million LoC for a rich chat server is absurd. To paraphrase Bill Gates, I don't think I could "spend" that many lines of code on a rich chat server if I tried.
Kudos not for simply calling attention to the problem, not for simply proposing an alternative, but for reminding people that the wheel need not be reinvented - cleverly implementing existing systems & subsystems to utilize their properties (e.g. ssh for authentication and encryption) in a manner that meets the intentions of those properties but not the originally imagined purpose of the system or subsystem is as good an example of the hacker ethos as any, and the software world would benefit immensely to draw from this concept. Lower costs, shorter development timeframes, and less engineering hours wasted rewriting functionally identical code is good for the developer, good for the company doing the developing, good for the end user, and ultimately good for humanity.
I wish to see a lot more of this kind of creative destruction and will try to implement such clever techniques in future projects myself.
This is generally a good ideas as it increases the speed you can ship & reduces the size of the org.
But there’s a big trade off: complexity & abstraction.
The higher level your abstractions the worse it gets too.
So it’s not without reason.
Indeed, multiplatform is very very costly.
[0] https://github.com/mattermost/mattermost/tree/master/e2e-tes...
[1] https://github.com/mattermost/mattermost/tree/master/webapp
I think the principal here is not reinventing wheels.
_any_ shebang script really... except perl scripts, because perl has taint mode turned on automatically when uid != euid
https://perldoc.perl.org/perlsec#Taint-mode
Note that there may be a fundamental race condition on some systems:
https://perldoc.perl.org/perlsec#Shebang-Race-Condition
Also see the noted use of `sudo -T`:
https://perldoc.perl.org/perlsec#Using-Sudo
I lament the recent removal of taint mode from Ruby :( as tainting is a powerful mechanism that could go way beyond scripts: imagine Rack / Rails #html_safe but via tainting so it could have worked more thoroughly (e.g concatenate/interpolate/format a tainted user string into a SQL snippet would blow up, preventing a whole class of SQL injections by design)
Some folks used to use taint for CGI/mod_perl to such great effect. It's not a magic bullet by any means (security is hard!) but it's a really effective tool still.
https://gitlab.com/edouardklein/suc/-/blob/master/suc_wrappe...
Until I found the wrapper file in the GitLab repo, I was holding out hope as I read through that there was some way.
https://adamdrake.com/command-line-tools-can-be-235x-faster-...
We all know "suc" is merely a tongue-in-cheek thought exercise. :) Good luck getting 100's of millions of Twitter/Bluesky/Threads users to even use Mastodon.
Nobody but geeks can use "suc", Mattermost can be used by a far larger proportion of people.
Good recently posted HN-linked article about how most users have a lot less skill than you think: "The Distribution of Users’ Computer Skills: Worse Than You Think": https://www.nngroup.com/articles/computer-skill-levels/
On any fair metric, this should inflate the volume of code metric for any project (whether leveraging those APIs directly or not). It would still favor this implementation on that front, and there are still other merits to the approach besides code volume. But something does irk me about touting supposed minimalism which externalizes almost all of its maximalism. Sure, the dependencies are probably there and sure, it’s good to use the platform. But it’s not a reasonable claim that “five lines” can recreate any portion of any moderately complex software. Otherwise we’d have “SSH in five lines of bash” and so on.
In scientific programming I'd say that's the same as using a library like GSL, BLAS or even numpy. The net impact on LoC in my project is minimal, even though it could potentially be calling thousands of lines of code. The point is that from a maintenance perspective I only need to maintain 5 or 10 lines, and if I find a bug in there I can file a bug report upstream, rather than maintain the complex details of the implementation.
The title is being a bit smart-ass for clicks and the author admits as much right at the top. But beyond that it's pretty great that they implementated a fairly basic version of slack with standard unix tools in a straightforward way.
Not saying this to be argumentative, only to emphasize the same conflicting dynamic I saw in the post: this is exactly the same rationale that people routinely lambast here about NPM and other sources of dependencies. It’s libraries and frameworks all the way down. I’m cool with that, I’m just not cool with picking and choosing when it’s cool without any particular principle.
I think having five lines of code (well, it's more like 50 of so reading the article) with some key and reliable dependencies is better for maintainability than having no dependencies but a substantially larger code base. As you point out, there's definitely limits, and npm's left pad, is odd and is even packages are obvious example where the added dependency is less maintainable than implementing the code directly.
any dingbat with a terminal can produce an npm library that you can use in your application, the level of quality control is basically zero
but it takes a pretty strong track record to get your software into coreutils, or really any base linux distribution
to put it kind of cynically, i think there is an enormous difference between relying on ssh vs. relying on leftpad, gatekeeping based on competence measured over time is i think actually important and good to do
Let’s compare with suc:
suc can implement Mattermost’s core features with 0.005% of the code. This is madness !"
Madness indeed! But that is also why this is such an awesome article! Upvoted and favorited!
For example, what combination of shell commands can I use to output the number of files in a directory?
Hint: it's a trick question, you can't.
`find . -type f | wc -l`
touch foo; ln foo bar
Is this one file or two "links" to the same file? ;-0
I think of it like an old-timey 'card catalog'. You have a bunch of tiny drawers filled with cards. Some of the cards are big and blank spacers with a prominent tab sticking above the normal top edges (Directory). Sometimes you have a card that points to another card elsewhere in the catalog (link). Sometimes you find the details of a specific book on a specific shelf (block data).
Point is, they are all cards. The comment essentially asked for a command to say 'how many cards between these two spacers'. It's a "trick" question as old as usenet to spring the distinction between link inodes and list of blocks inodes and say "Ah-HAH!! gotcha", but in reality it's a silly game of jumping levels, misdirecting semantics and prey upon the distribution of understanding in a forum for personal glory.
The inode is the item, it is the card that is being counted, no matter what is printed on it. imho.
touch first$'\n'file
touch .hidden$'\n'file
mkdir folder$'\n'.
touch folder$'\n'./another$'\n'file
Your solution incorrectly outputs 7.
Oh.
Then it must be:
command ls -F directory | grep -v '[/@]$' | nl | tail -n 1 | cut -f 1
touch first$'\n'file
touch .hidden$'\n'file
mkdir folder$'\n'.
touch folder$'\n'./another$'\n'file
mkdir normal
touch normal/whatever
Your solution incorrectly outputs 3.
find . -type f -depth 1 -print0 | tr -d -c '\000' | wc -c
I had to modify it a bit to get it to work cross-platform, but the approach does appear to be sound:
find . -maxdepth 1 -type f -print0 | tr -d -c '\000' | wc -c
Whereas large scale enterprise commercial solutions make money by selling you on complete solutions that force you to relearn everything their way from the ground up. Just look at how AWS has hijacked so many concepts from the modern web and trapped people into building “cloud-agnostic” wrappers to try and wrangle the mess. Still we’re mapping onto their redesigns of the same old stuff.
At some level it’s unavoidable, hardware vendors need to agree on instructions after all, but when looking at the situation from a high level, it’s best to keep our ideals in mind and steer the ship so-as to find ourselves in paradise not lost in a sea of pirates.
One ideal: Modularity. Promotes healthy competition. Another ideal: Simplicity is more easily achieved when modular components may be used (and tested) in isolation.
* Roll it themselves, maintain the database and all that jazz * Delegate it to the (very robust, very mature) Linux user authentication stuff
Ever since reading that I've found myself wondering why more apps don't simply use SSH keypairs for authentication, given that they're already such a battletested mechanism. I get the whole "no MFA!" argument, but still.
However, is there a race condition with writing to the files? Since it is append mode, I would expect that would prevent other processes overwriting the file, although would it prevent other processes writing in the middle of a line if it gets interrupted? Actually, I don't know.
I had made a simple two user chat system with logging using ts, tee, and nc; using a shell script with only a single line of code. This produces two log files, one for sending and one for receiving; however, I can then use cat and sort to interleave the logs into a single log file.
I don’t enjoy Slack and the like due to the notification overload and FOMO burnout. The UI is a memory hog for what it does.
But for the computer user that wasn’t born at Bell Labs with a keyboard in their hands, it’s pretty decent. If you can navigate gmail or a word doc you can probably figure out how to use Slack.
... and over 50 years of operating systems development.
The authors are right when they say that sshd and unix users are probably the most frature-full auth scheme in existence.
Too many software project just ignore the underlying abstractions of the OS they run on. I feel weird with all this virtualization stuff, like we're missing something obvious.
Rob Pike said in 2000 that systems research is irrelevant http://doc.cat-v.org/bell_labs/utah2000/ and I'd like to think that if we went back to simple tools like this we could make it somewhat relevant again.
Lots of unix facilities are criminally underutilized in modern systems
I don't know how much effort is meant to be put into securing this from impersonation, but info can be pulled out of /proc/ if the permissions are set up right.
bash is concise but there are other languages nearly as concise. It'd make more sense to use them rather than go+bash, once you cross the rubicon of not relying entirely on stuff that comes with UNIX out of the box.
$ rain | wall #.. my favourite thing to do to my colleagues during the end of coding marathons we'd find ourselves in, during the 80's ... on machines that weren't always fortified against the rain .. ;){We would often get people using lab computers remotely and you could talk or wall them to tell them you were rebooting soon. Good times.
Disclaimer: Have a "The Wall" account somewhere. What goes around, comes around ..
cat /dev/audio | <( mkfifo && netcat )
(whoops, forgot the gzip/gunzip filters...)
https://bbs.archlinux.org/viewtopic.php?id=43896
(circa 2008 people were doing this... semi-unironically ;-)
It was solved over 40 years ago.
That's the thing with social networking. There's a few core problems and it's been solved many times. Such as in 1973 at Community Memory in Berkeley: https://en.wikipedia.org/wiki/Community_Memory
Or through newspaper classifieds ads, French salons, compuserve, bars, aol, cb radio, friendster, phone phreaking party lines, dialup bbses, icq, irc, myspace, Facebook, netnews, whois/finger/talk/uucp. If you read Carolyn Marvin's book, "When old Technologies Were New" (1988) you'll see her documentation of this in its first digital form - via lonely telegraph operators chatting with each other in the 19th century.
It's the network, not the medium, that matters.
: rm -rf ~/
They cannot. On careful rereading usuc passes the data through the pipe when writing a message, not reading it. So the channel is just full of raw terminal escape codes, if I understand correctly.
With that said, I seldom used IRC so this is a genuine question for people who used it and miss some features.
At the risk of becoming a meme, it's similar to how practically nobody uses Linux alone; it's almost universally accompanied by something like GNU.
For some, IRC is the example why slack/discord won.
What options are there and what are your experiences?
while read -r line ; do printf '%(%FT%T%z)T %-9s %s' -1 "$(/usr/bin/id --user --name --real)" "$line" ; done
Using builtins may be a security risk as they can be overloaded (hence the use of full paths everywhere).
I did not know about bash's extension of printf to print a date. I need to use the builtin to use it though. But I've received good advice elsewhere on how to do that securely, so I'll do it and do a write-up because it's not information that's easy to come by.
Your advice is very helpful, thank you.
function /usr/bin/cat { echo hello; };
/usr/bin/cat
hello
Although it is technically cool, it misses practically all of what makes Slack so popular - UX.
I strongly disagree. Slack has nearly the same UX as every chat platform of the past 2-3 decades. It's a slightly change over AOL Instant Messenger. There's also a large amount of very similar software or straight-up slack clones that are not very popular at all.
I would say what makes Slack so popular is how easy it is to set up and get running, both as a service and for every user, combined with the availability of easy integrations/plugins.
That is part of the User eXperience. I.e. UX.
1. being connected to multiple communities and switching between them instantly
this can be of course simply replaced by connecting to different servers in a tabbed terminal and use the terminal's built-in cmd-1/2/... shortcut, which happens to be the same as in slack.
2. meta data about others, like their timezone or how to pronounce their name is quite important for distributed team work
this can be approximated by a world readable file on the chat server in every user's home, like .plan or motd files (https://github.com/ESWAT/john-carmack-plan-archive)
3. automatic idleness detection
im actually not sure how reliable is this even in slack, but in general, it can be useful, but im not sure how to solve it elegantly, when the chat runs remotely...
maybe we should just spawn a loop at the background, which gathers idleness status from the OS and uploads it when it changes, into world readable files and the remote clients can just check those file whenever they want.
4. extra status indication with automatic expiry, eg when someone is away from the keyboard, coz they are having lunch
we do use this feature often and it's a really helpful regarding when can we expect a response from someone.
again, quite simple to model this as a plain text file and we can even use emojis, to have a very similar effect to setting " lunch" on slack. ppl would need to know what's the emoji selector shortcut though... like cmd-ctrl-space on macos.
5. text search across all channels/rooms
assuming the chat is being logged into files, then a recursive (rip)grep could work to some extent, but then from the search results one might want to get back to the context of the result too.
6. threads
this complicates implementation a lot more, but we found it an obvious improvement over the single threaded IRC model of communication
7. having threads open on the side, so ppl can track 2 streams of comms at once at least
it would require starting the chat app multiple times and do some window management to see them side-by-side
now obviously all this can be done a lot simpler, but those implementations typically always lack somehow. not sure why is that...
see https://cancel.fm/ripcord/ or http://www.altme.com/what.html
the REBOL 3 programming language even had a quite full featured, text-mode chat built in: http://www.rebol.com/r3/docs/functions/chat.html !
_proceeds to sign up for slack._