Hacker News Reader

> The problem with a queue on SQLite is that every successful read implies a write.

SQLite also supports in-memory databases with shared cache.

https://www.sqlite.org/inmemorydb.html

If a message queue does not require any form of persistence, writes don't sound like an issue.

But this is purely in-memory with no disk persistence. That’s quite risky

Not to be coy, but it only is if it is. For this application I'd agree, but there are plenty of apps that want queues for in-memory use, and if the thing that holds the memory for the queue dies, chances are the whole ecosystem has.

Sometimes this is fine.

We used sqlite for a queue at a prior startup - enabling the WAL was critical to maintain disk throughput.

I once saw WAL turned off to make a sqlite queue perform better.

But that was ... gremlins. More than one person tried to figure out wtf was going one and eventually it was better business wise to declare 'gremlins' and everybody involved in the incident has been annoyed about not figuring it out since.

I just did a quick test, I am seeing over 1k commits per second when being driven from Python 3.11 on an old macbook pro.