Hacker News Reader

Comment by indymike | Hacker News Reader

I once got hired to fix an application where the original dev heard "message queue" and implemented in using email boxes because... wait for it... because email has queues full of messages. So, instead of reaching for an actual message queue, everything was queued via emails, and worker tasks started by fetching an email message. About all I can say, is, it worked... sometimes slowly.. but it did work, and had the benefit of allowing you to inject stuff into the queue by sending in a properly formatted email.

View on HN

Thinking strategically about the most reliable systems in our organization today, email is probably at the top of the list. I feel like there is a semi-auto magic thing email is still missing for a lot of businesses. Many things can be managed within the latency constraints of email. All of the hardest & most complex things almost certainly can be. For instance, on-boarding a new employee could be totally driven by some back-end service that simply sends & interprets emails all day to ensure that all of the required activities are executed.

Now I am wondering if I can rewrite our B2B product interface to just be email. The more I think about this the more it starts to make a ridiculous amount of sense...

  User=>Server (Email 1):
    Subject: Get Workflow List
  Server=>User (Email 2):
    Subject: Workflow List [hash code]
    Body: <all workflows by name and id>
  User=>Server (Email 3):
    Subject: Start Workflow #21
  Server=>User (Email 4):
    Subject: Workflow #12345532
    Body: <First page of onboarding questions the user needs to fill out>
  User=>Server (Email 4 - reply):
    Subject: Re: Workflow #12345532
    Body: <First page of questions with responses>
  Server=>User (Email 4 - reply):
    Subject: Workflow #12345532
    Body: <Second page of questions, or error regarding first page responses>
  ...
  Server=>User (Email 4 - reply):
    Subject: Workflow #12345532 [COMPLETED]
    Body: <Completion summary with whatever final artifacts attached>

I know our users would totally reject this proposal if it were made to be the only way to interact with our product, but if it were an additional way to interact, they might grow accustomed over time.

Think about the technical advantages of the above. If you keep it plaintext, anyone with a computer made within the last 3 decades could access your business system unconditionally. Developer productivity is likely excellent. I can't imagine you would be able to get very distracted with plain text emails.

As it happens we have an onboarding platform which certainly emits quite a few emails.

We use online forms for data capture, but funnily enough things sometimes pan out like you say.

It seems that if you send someone an email saying "You forgot to upload your drivers licence, please click _HERE_ to upload it", a good 50% of the time they will simply reply to your email, attaching the document. And why not really. We used to try again to direct them to the online system, but eventually started handling these things outside the usual flow.

tldr; your idea is sound!

The email server is very reliable, in the same way that an Apache web server or MySQL database is reliable.

It's the script pulling data over the network, parsing it, developers updating it, etc, where the instability comes in. And don't forget the edge cases where somebody uses weird encoding, a large email with attachments, a client that formats it in weird ways, etc.

I probably took your comment way too seriously.

Why not chat interface? This kind of communication you just described feels unnatural for email, but something people would do over chat/messenger in a heartbeat.

You laugh, but Active Directory supports SMTP as one of its transport options, because it allows occasionally-connected sites to participate in replication.

Well, email is by orders of magnitude the most hardened and most widespread message queue system out there. It’s like, it seems stupid... until it doesn’t.

This is something I can definitely see myself doing, even if I might regret it later. Now that I've got code for sending emails, it's only logical that I write code to pull them back down again and...

I have seen that too in a previous job! It worked until there was a restart in the e-mail relay server, or something in the message (I think it was Java, sending multipart messages with some custom message format).

ServiceNow had (still probably has) something called inbound actions.

This is people clicking on mailto: hyperlinks in servicenow notification emails to do stuff like approve/reject a service request or reopen an incident ticket.

You can go beyond this functionality and build any workflow to be triggered after consuming an inbound email sent to client@servicenow.com

I am sure there are a number of email based integrations built using this especially when the upstream application didn't speak API but could do email.

Potentially in original dev's defense, there was a time when OTC queue libraries, like ActiveMQ, weren't usable.

During the rise of J2EE, long before Jespin, we were directed to use JMS. Because architecture. All of the products we tried for our trivial use case would ABEND or break FIFO.

We submitted bug reports, repo steps, and patches to ActiveMQ. Closed, WONTFIX.

So I gave up. Nobody has time for all that.

Motivated by postfix, I just wrote inbound messages to disk. File name included timestamp and counter. fsync() for the win. (What more could we do?)

Processing a queue meant moving a file to a different directory. Logging, archiving, and grooming were just cron jobs and shell scripts. Tada!

Like your predecessor's solution, this allowed trivial testing, observation, and admin.

I wouldn't do this again today for anything high volume. But for less than 100s of messages per minute? Probably.

Reply on news.ycombinator.com