anitil

I mean.... it does sound like a fun project

There is a way to do this with sqlite in readonly mode using the append vfs [0] which allows you to put your DB appended to the end of the executable, and then the executable can read it (like a zip file the header is then at the end of the db rather than the front). Or you could embed a normal db as a binary blob with the linker. Allowing writes is more tricky because most OSes don't typically allow executables to edit their own executable memory (on unix I believe executables always load as r/x or r/o depending on the region, maybe with some minor exceptions).

And a lot of users of sqlite statically link sqlite within their executable, they actually recommend that instead of dynamically linking.

[0] https://sqlite.org/vfs.html ctrl-f for 'append'

Edit: I got confused between the sqlar command and the append vfs so fixed it.

Huh TIL! I've spent a lot of the last few weeks reading sqlite docs and somehow either never saw that or forgot it

There's a really good list of lessons in the second half of this page which I think are really interesting (in general, but also in specific for this project). I don't have a Lobste.rs account but I have some comments about some of them so I'll add them here

* If you want to fail on full table scans in tests you could take advantage of EXPLAIN - though, because it relies on the built in analysis/stats this may exactly mimic production tables. I believe Ruby is dynamic enough that you could probably attach an EXPLAIN to all queries in the unit tests [0] and parse the output.

* For the three UDFs - Sqlite has a regexp module that you can build and load rather than using a udf in rails (probably a different version of regexp though, so user beware) [1] there is also an extensions_functions.c in the external contributions that provides 'stddev' [2] (or you could fake it with the percentile module). I'm confused why there's a udf for 'if' but it'd be trivial to build as C module, but I agree it's best to patch over this sort of thing separately from the migration, so perhaps limited benefit.

* 'I'm constantly surprised by the default choices of SQLite.' I think this is fair, but they're quite dedicated in not breaking existing installations, they even have pages dedicated to explaining some peculiar architecture [3]

[0] https://www.sqlite.org/eqp.html EDIT: there's an alternative suggestion here - https://lobste.rs/s/ko1ji1/lobste_rs_is_now_running_on_sqlit...

[1] https://sqlite.org/src/file/ext/misc/regexp.c

[2] https://sqlite.org/src/ext/contrib/ (ctrl-f for 'extensions-functions.c'. It has more than just stddev but you could strip the rest)

[3] https://www.sqlite.org/rowidtable.html I find this a really touching note 'The designer of SQLite offers his sincere apology for the current mess'

I mis-read the title and thought someone was going to pay 65k to work at Anthropic. And ... That's a strange thing to do, but financially would probably pay off in the end

I've been building some sqlite plugins for playing with ngrams for text search. I'm not sure why, but I've learned a lot about the internal sqlite apis and it brings me a lot of joy. I would like to start a blog detailing some of this work but haven't found the time yet

I noticed that it wasn't the best comment, I was only concerned with the tone, and I feel like dang has enough going on that we also need to help elevate the conversation. I admit there's some delicious irony in the accuser committing the same crime, but it doesn't improve the discussion to revel in that.

Your reply would have been much better without the first line [0]

> Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that"

[0] https://news.ycombinator.com/newsguidelines.html

Very cool story. I'm interested to do a similar transition, though I was thinking it would be mecessary to go back and do a masters. It's interesting that that might not be required

This was equal parts interesting and terrifying (both from a ecosystem and from a 'will I have a job next year' perspective)

I do the same. But the benefit to me is that by the time I've written out the message and checked my work I've usually solved the problem. So it's like rubber-ducking but with social consequences attached which makes my brain actually pay attention.

The latest post from the very interesting exe.dev project (previously of 'I am building a cloud' [0]). They have a lot of interesting work that goes in to making SSH work with their system (eg previously [1]), and this is a continuation of that.

Using Unix sockets to transfer file handles is not a new idea, but using for SSH connections is interesting to me. I wish it were a little heavier on the details here, but otherwise this is an interesting approach to handling network traffic while maintaining zero-downtime upgrades

[0] https://crawshaw.io/blog/building-a-cloud

[1] https://blog.exe.dev/ssh-host-header

Always a good day when I get to look at the McMaster catalogue

The article mentions that you typically have to longjmp within the same function as setjump (or a descendant function) otherwise your stack gets cleared and you longjmp to a garbage stack. I believe this counts as memory safety? Though I don't quite understand your comment about sigaction, so maybe there's some context I'm missing.

Edit: The extra context- https://usenix.org/legacy/publications/library/proceedings/u...

How interesting! I thought that setjmp and longjmp were probably incompatible with Fil-C. And I'd somehow never heard of ucontext at all.

I suppose managing the stack is still managing memory after all, even if we typically don't think of it that way, so Fil-C has something to add here.

It's really worth reading the section here about the complexity of setjmp/longjmp and how they interact with register allocation and stack spilling. I knew they're tricky, but going in to the specifics is delicious.

> writing well is really hard

I think because good writing is easier to read and understand than bad writing, there's almost an inverse relationship between how easy writing seems and how easy it is to read. So you read something and think "well, this is obvious" but the magic trick is how much work went in to making it so.

Interestingly the earliest capacitors were glass jars (called Leyden Jars) [0]. I was taught that early inventors thought that charge was accumulated within the jar, and it wasn't until much later that it was realised that the shape was irrelevant, only the area and the distance between conductors.

[0] https://en.wikipedia.org/wiki/Leyden_jar

And sometimes you click a permanent action ('ignore' or 'accept' or 'decline') out of muscle memory or an imprecise click. Just this morning I _think_ I allowed some telemetry on a system I use (but don't want telemetry on) because by the time I had registered the text I'd already clicked it and now I can't work out which setting I need to toggle back

My go-to for this is to screenshot and use the built-in text extraction in the screenshot tool (I'm on a mac), then pass on that text data to whatever processing. It's a pretty good tool so long as the PDF is in OK shape (I've had errors in scanned images).

I feel like any problem that Postgres can't handle is a good problem to have. Either you've got so many customers that you're hitting sharp edges, or you're working on such an interesting problem that you're out of the domain where Postgres is helpful. That I should be so lucky

The latest in the 'Mars for the rest of us' series that covers a host of issues with space travel. This one covers the issue of food and nutrition in space.

Including this charming little tidbit -

> Officially there is no alcohol allowed on board, but every once in a while NASA spends a few months trying to run down a spike in atmospheric ethanol that is inevitably traced to obscure origins somewhere in the Russian half of the station.

This was a lovely tribute. As much as we like to complain about windows, there's clearly a lot of love and care that has been put in to it by people like Tony

Interesting, I was wondering where catching these errors would fall between 'silently wrong on certain inputs' to 'how did this ever work!?'

Do we have a sense for how many of the programs that work [0] are now detected as having asm violations?

[0] https://fil-c.org/programs_that_work

I find it charming that to distinguish Fil-C from the K&R language they use the term 'Yolo-C'. I have never used inline asm before, I actually didn't realise how much behaviour it's specifying! (When I've needed asm it was on non-gcc compilers)

Edit to add: If I'm understanding this correctly we should be able to run this against projects and detect asm violations, I feel like this would be very valuable to be able to feed these back to maintainers

This is very cool! I've wanted to do something like this for Sydney but never got around to it. Are you scraping your data sources or building them yourself?

I think you might be right, that rings a bell. There's an item on the front page about this right now, and I'm guilty as charged - I don't understand CORS at all

https://news.ycombinator.com/item?id=48614844

It's probably out of scope for you, but I've used the 'vtfunc' module [0] for a similar purpose actually.

[0] https://github.com/coleifer/sqlite-vtfunc

Interesting idea, I know there's the fsdir [0] table-valued function / module that allows loading from disk, so it should be possible to modify that or hard-code base list of paths or something

[0] https://sqlite.org/src/file/ext/misc/fileio.c, it allows you to read a directory recursively in the cli (`select * from fsdir("./");`)

Edit: It allows upwards traversals (`select * from fsdir("../../../../etc/passwd");`), so beware

DuckDb makes so much of my life easier, though I've never used it for large problems. The ability to run `select * from 'data.json'` is just lovely. The fact that it's also a powerhouse is so impressive, I'd usually expect a project to be good at small problems (like mine) xor large problems, but not both