matharmin

As mentioned in those threads, there is no SQLite WAL corruption if you have a working disk & file system. If you don't, then all bets are off - SQLite doesn't protect you against that, and most other databases won't either. And nested transactions (SAVEPOINT) won't have have any impact on this - all it does in this form is reduce the number of transactions you have.

We're relying on logical replication heavily for PowerSync, and I've found it is a great tool, but it is also very low-level and under-documented. This article gives a great overview - I wish I had this when we started with our implementation.

Some examples of difficulties we've ran into: 1. LSNs for transactions (commits) are strictly increasing, but not for individual operations across transactions. You may not pick this up during basic testing, but it starts showing up when you have concurrent transactions. 2. You cannot resume logical replication in the middle of a transaction (you have to restart the transaction), which becomes relevant when you have large transactions. 3. In most cases, replication slots cannot be preserved when upgrading Postgres major versions. 4. When you have multiple Postgres clusters in a HA setup, you _can_ use logical replication, but it becomes more tricky (better in recent Postgres versions, but you're still responsible for making sure the slots are synced). 5. Replication slots can break in many different ways, and there's no good way to know all the possible failure modes until you've run into them. Especially fun when your server ran out of disk space at some point. It's a little better with Postgres 17+ exposing wal_status and invalidation_reason on pg_replication_slots. 6. You need to make sure to acknowledge keepalive messages and not only data messages, otherwise the WAL can keep growing indefinitely when you don't have incoming changes (depending on the hosting provider). 7. Common drivers often either don't implement the replication protocol at all, or attempt to abstract away low-level details that you actually need. Here it's great that the article actually explains the low-level protocol details.

Probably a security feature. If it can access the internet, it can send your private data to the internet. Of course, if you allow it to run arbitrary commands it can do the same.

FoundationDB also has a Mongo-compatible document layer, but it seems like the last release was 6 years ago, so probably doesn't count anymore.

The project looks great! Object storage is often so much better in terms of cost efficiency than a database on EBS. It's often 10-20x more expensive for EBS after taking into account that you need 3x replicas for a typical MongoDB deployment, and need to over-provision the storage. And being able to scale compute independently from storage is great.

The biggest things I'm missing from the docs (checked the github page and the site) is seeing what MongoDB features are supported or not. I've worked with Azure CosmosDB before, and even though it claims MongoDB compatibility, it has many compatibility issues as soon as you have more than a basic CRUD application. Some examples include proper ChangeStream support, partial index support, multi-key index support, set of supported aggregation pipeline operations, tailable cursor support, snapshot queries.

Another thing that's not clear: What does multi-master/multi-write mean in practice? What happens if you write to the same data at the same time on different nodes?

That is completely wrong for this stage of a company. The ability to make profit in the future is important. Making profit while growing is not.

Let's say I have an open-source project licensed under Apache 2. The grant allows me to include the extension in my project. But it doesn't allow me to relicense it under Apache 2 or any other compatible license. So if I include it, my project can't be Apache 2-licensed anymore.

Apache 2 is just an example here - the same would apply for practically any open source license.

The one place I imagine it could still work is if the open-source project, say a sqlite browser, includes it as an optional plugin. So the project itself stays open-source, but the grant allows using the proprietary plugin with it.

It would help to have more info on what you need to run this. The page mentions "without the need for any new hardware", but doesn't say what existing types of hardware it is compatible with. The apps available for download gives a hint, but are the apps for displaying the content, or for controlling the content?

For example, I recently setup a dashboard using a Raspberry Pi running Chromium - would this work for my use case?

And does the control work over the local network, or does it require an internet connection?

> Overall, I haven’t seen many issues with the drives, and when I did, it was a Linux kernel issue.

Reading the linked post, it's not a Linux kernel issue. Rather, the Linux kernel was forced to disable queued TRIM and maybe even NCQ for these drives, due to issues in the drives.

There may be good reasons to do that, but this isn't one. Any project using CocoaPods will still remain working for the foreseeable future - you'll just not get new updates to dependencies at some point. And at that point you can migrate to SwiftPM or vendored dependencies, without losing anything.

You'll still be able to use those - the CocoaPods repo will not go away any time soon. If someone wants to provide updates for those, it has to be migrated to SwiftPM. And in the meantime you may have to use both SwiftPM and CocoaPods, or fork & migrate those yourself.

wa-sqlite on top of OPFS is actually pretty great these days. Performance is about half of what I'd get in native SQLite, which is not too bad overall. It's around 10x faster than SQLite on top of IndexedDB for large databases in my experience.

It's much better than WebSQL could ever be. You get the full power of modern SQLite, with the version, compile options, additional extensions, all under your control.

Luckily there are still other options out there. ZeroSSL is one I quite like: Free ACME-based certificates just like LetsEncrypt, without the rate limits, and does have paid plans if you need support. It also has better legacy client compatibility than LetsEncrypt as far as I know.

Certificates have a static expiry date by design - it's not "LetsEncrypt expiring the cert". There is no way to avoid expiring a cert if the DNS entry is still there - all you can do is make it easier to renew the cert. That means it must be automated, in which case it doesn't matter if you need to re-create a DNS entry.

In my experience, it takes a little effort to set things up the first time, but from then on it just works.

Destructors I other languages are typically used for when the object is garbage collected. That has a whole bunch of associated issues, which is why the pattern is often avoided these days.

The dispose methods on the other hand are called when the variable goes out of scope, which is much more predictable. You can rely on for example a file being closed ot a lock released before your method returns.

JavaScript is already explicit about what is synchronous versus asynchronous everywhere else, and this is no exception. Your method needs to wait for disposing to complete, so if disposing is asynchronous, your method must be asynchronous as well. It does get a bit annoying though that you end up with a double await, as in `await using a = await b()` if you're not used to that syntax.

As for using symbols - that's the same as other functionality added over time, such as iterator. It gives a nice way for the support to be added in a backwards-compatible way. And it's mostly only library authors dealing with the symbols - a typical app developer never has to touch it directly.

In my opinion neither hooks nor CI should ever make changes to code automatically. When I commit changes, I want to see exactly what I commit, and not have some system change it at the last minute.

Instead, have tooling to do that before committing (vscode format-on-save, or manually run a task), then have a pre-commit hook just do a sanity-check on that. It only needs to check modified files, so usually very fast.

Then, have an additional check on CI to verify formatting on all files. That should rarely be triggered, but helps to catch cases where the hooks were not run, for example from external contributes. That also makes it completely fine for this CI step to take a couple of minutes - you don't need that feedback immediately.

This doesn't work if you have branch protection rules blocking pushes to main, which in my opinion should be standard on any repo

I view JQuery as similar to C in some ways: It's been around forever, it's mature, and it works. It gives a good experience to get something up and running quickly: it's lightweight and simple.

But if you're working on bigger projects: It is possible, but have have to be very principled in how you use it, otherwise you're going to end up with either a massive spaghetti codebase and lots of edge cases in your app that breaks.

Alternatives like React and Rust may add more complexity upfront, but the improved structure and safety it gives has big benefits in all but the smallest projects.

I think I'm missing something here - what is specific about Docker in the exploit? Nowhere is it mentioned what the actual exploit was, and whether for example a non-containerized postgres would have avoided it.

Should the recommendation rather be "don't expose anything from your home network publically unless it's properly secured"?

With your own native client: Yes, you can send arbitrary headers in the Upgrade request.

In a browser however, you can't. It typically sets very little headers itself, and doesn't allow you to add custom headers.

It feels a little misleading to say you're using Google Sheets as the backend database, when you need an actual database in conjunction with it.

In my experience, the "outbox" approach means a lot more manual work for developers. It requires developers to create a message for every type of change they want to sync to the client, and then also interpret that on the client. ElectricSQL's Shapes does a lot more work to keep the shape in sync between the client and the server, reducing the need for the developer to do that work.

You're right that "single-table sync" does have its limitations. At PowerSync we effectively support one level of "joins", and even then it's often not enough for more complex schemas. An older version of ElectricSQL did also actually have multi-table shape sync support, but I believe doing that at scale proved to be difficult.

One solution to this is often denormalizing data - either adding more denormalized columns in the existing table, or creating new tables dedicated to sync data. Conceptually, keeping these tables up to date is not that different from writing updates to an outbox table.

I'm also interested in seeing what Zero comes up with in the space. They seem to have solved doing multi-table query sync, but it remains to be seen how well that works in practice.

They did not say it was dementia, and they did not offer care suggestions - they merely shared their own related experience.

Unlike other opinions here I do think it is technically feasible to stream a copy of the WAL - it just has to be implemented in the VFS. "Shared memory" could be a SharedArrayBuffer, or just a normal buffer if you only have one database connection open at a time (for example in a SharedWorker, which is already common). It may not be simple to implement, but definitely possible.

The biggest actual issue is that it will capture block-level changes, not row-level changes. This means it can work to replicate a complete database, but partial sync (e.g. sharing some rows with other users) won't be feasible.

To get row-level changes for partial sync, you need to use something like triggers or the SQLite session extension [1]. For PowerSync we just use triggers on the client side. I find that works really well, and I haven't found any real downsides to that except perhaps for the work of maintaining the triggers.

[1]: https://sqlite.org/sessionintro.html

Just wondering, do you have a specific use case for read transactions implemented on the database level here?

In SQLite in general read transactions are useful since you can access the same database from multiple processes at a time. Here, only a single process can access the database. So you can get the same effect as read transactions either by doing all reads in one synchronous function, or implement your own process-level locking.

I can't tell whether you're trolling or serious...

It's not about moderators being harmed or not. It's a preference of the wider community to not have the harmful content, and the moderators volunteering to help keep it that way.

With most harmful content, the effect of seeing a single instance is not significant. Being exposed to it constantly in places you frequent for other reason (your online communities, or advertisements everwhere), builds up much more of an effect. Now moderators being constantly exposed might also be affected, but they're choosing to do so, and may have coping strategies in place for the more extreme cases.

By your argument, moderators on HN/Reddit shouldn't remove harmful content, since moderators would still have seen it either way.

There is a proposal for that: https://github.com/tc39/proposal-type-annotations

It sounds like a niche use case where SQLite does work quite well server-side without needing any replication, since the database is read-only.

Other alternatives may use static files loaded in-memory, but I'm guessing the data is more than you'd want to keep in memory in this case, making SQLite a nice alternative.

As a novice with Rust maintaining a cross-platform library written in Rust, I've found panics/unwinding is one of the most annoying things to figure out in the builds for various platforms.

I'm using nostd and panic=abort to keep the library small (around 200kb after being stripped), but it's really not a well-supported setup.

Depending on the platform, I get an error that the _Unwind_Resume symbol is not defined (despite it not supposed to be needed), or if I define it myself I get a duplicate symbol error on other platforms. I ended up linking to gcc_eh on Linux just to avoid the issue.

I just find it strange that out of all the possible issues with using nostd, unwinding is causing the most issues.