matharmin

This feels like a marketing piece without any good content.

The way it's described, it's not a knapsack problem at all. The knapsack problem is to maximize the total value of the items you fit into the container.

In reconciliation, you presumably want to get the best matching between transactions, which is not defined here, and in any case is a completely different problem.

Ignoring the knapsack comparison, the article doesn't describe why you'd want to check each possible combination. Assuming the individual amounts are correct, you can do each batch separately - no need to check each combination within one batch with each combination of a different batch. (And if you drop that assumption, that still won't be a sensible thing to do).

I can imagine you can have a "scoring" algorithm that gives a confidence score for a match - then if you check every combination, you can pick the combination with the best overall score. But the article doesn't actually describe anything like that.

It also doesn't describe any alternatives to "AI". For example, what about a greedy algorithm? What about alternative methods to do address comparisons? I'm sure there are issues with those, but none of that is described here.

Reading their docs: No real benefits, just misconceptions.

1. Collision resistance / "weak" PRGNs used to generate UUIDv4. Firstly, these are properties of the implementation, not the spec. Secondly, the source for calling the browser `Crypto.getRandomValues()` insecure is an issue that has been fixed back in 2016. I would not trust the developers of this implementation to do a better job than current browsers.

2. "Not URL or name friendly": Fair, but not very strong argument.

3. "Horizontally scalable" and "offline-capable": No argument given for why UUIDv4 does not meet these requirements, apart from point 1 above.

4. "Too fast": No argument given for why having a slower algorithm to generate random ids is more secure. Both UUIDv4 and Cuid2 use a similar number of random bits (122-124). When using a secure PRNG, both are equally difficult to guess, the SHA3 hashing doesn't add anything. You don't have to try and guess the "input" of the Cuid2 - you can just try to guess the "output" and skip the SHA3 hashing. It would be impossible to actually guess a generated ID, but UUIDv4 is just as impossible. Also no argument given for why UUIDv7 is fine but UUIDv4 is not.

I've used UUIDv4 for genering unique IDs for over 10 years now. I have run into collisions, when I hand-rolled my own implementation for J2ME with major bugs many years ago - ended up with around 20 bits of entropy instead of 122. That's not a reason to not use UUIDv4, just a reason to not implement it yourself unless you really know what you're doing.

You have good points overall, but I'd say AngularJS did have mass adoption at a time - it was a great way to build web applications compared to the alternatives. React just came by and did even better.

And Dart may not have much of a standing on its own, but Flutter is one of the most popular frameworks for creating cross-platform applications right now.

This is a great article. I've been working on tooling related to local-first/offline-first for the last 10 years, and the benefits both for end-users and developers are great.

The biggest benefit for developers that I've seen is that state management gets _much_ simpler when it is all handled by a local database. Just run queries directly on every app/page, with live queries if the framework supports it. No need for maintaining in-memory app state and propagating changes to the different parts, no need for caching and associated cache invalidation issues.

The caveat is that keeping the local database in sync with the server is not easy, and it's best to use other tooling for it. I'm working on PowerSync [1] myself, and ElectricSQL [2] also looks quite promising (both also mentioned in OP).

[1]: https://powersync.co

[2]: https://electric-sql.com

"for await" uses async iterators, which is pretty much the same thing as a stream - just way simpler to use.

SQLite is quite good at staying backwards compatible, and this includes the APIs exposed to extensions, but extensions could definitely have a minimum SQLite version.

Where I live, this is actually extremely common now - typically referred to as a "Mini DC UPS" [1].

These cheaper ones just provide 9V and 12V output to power a fibre and/or WiFi router, as well as 5V USB-A, and start at around 30Wh capacity - some for less than USD 20. If you pay a little more, you can get 24/48V PoE included as well.

Unfortunately, ones providing USB-PD are quite rare. You do get "power stations" [2] with USB-PD (in addition to an AC inverter). These are typically USD 100 - USD 1000, and have much larger capacity.

Examples (I haven't tested these and do not specifically recommend them):

[1]: https://www.amazon.com/Lithium-Battery-Appliance-Monitor-uni...

[2]: https://www.amazon.com/Portable-Solar-Panel-Power-Station-Ge...

SQLite is really great database for mobile applications, where concurrency can make a massive difference in how responsive the app is.

Just as one example, imagine a mobile app that does a bulk data download, and needs to persist it all to the local database in a single transaction for consistency purposes. This transaction can take a couple of seconds, or maybe a minute of you're working with millions of rows.

Now without concurrency, your database is locked for the entire transaction, and this may block any user actions.

With WAL, you can at least continue doing reads during this transaction.

With BEGIN CONCURRENT, the user may be able to continue most actions, as long as it doesn't touch the same set of data.

Now there are other ways of solving this problem: use smaller transactions and different mechanisms to ensure consistency. Or use separate databases if you can structure your data that way. But BEGIN CONCURRENT can be a nice solution without having to restructure the data completely.

Now these data volumes may be extreme, but even with "background" transaction taking in the order of a second can have an impact on how responsive the app feels to the user.

OP here. Persisting SQLite data in a browser is quite a complex topic in a rapidly-evolving space, with info spread out over official SQLite documentation, forum posts, GitHub issues and discussions. I spent many hours researching the various options and considerations, and I'm hoping this write-up will be useful to others.

DNS is typically a single round-trip for CNAME + A. And if you're using a resolver such as 8.8.8.8 or 1.1.1.1, it will route to a resolver close to you in most cases, and already have the results cached for any major site.

Main reason is that it's not in the standard release yet. This means it's more effort to get a build that supports it (build it yourself essentially), and it's not tested as well.

Additionally, most current tools will not be able to read database files using WAL2, although this is not an issue for most applications that only use it for internal storage.

In my experience, SQLite has excellent performance if you use it correctly. It can easily do 10k to 500k inserts/second, depending on the workload.

SQLite only supports a single writer at a time, but does support concurrent reads. There is also an experimental branch that supports concurrent writes.

Where I am, old electric organs are available for very cheap. I got an old Yamaha Electone a couple of years back, that wasn't working anymore. It was great fun to use an Arduino to read the keys and effectively convert it into a MIDI keyboard, and use an old PC with organ simulation software.

Unfortunately, I didn't have the space to keep it, and had to get rid of it eventually. Some day I if I have more space, I might do that again.

1. Use WAL mode - this allows reading while practically never being blocked by writes. 2. Use application-level locks (in addition to the SQLite locking system). The queuing behavior from application-level locks can work better than the retry mechanism provided by SQLite. 3. Use connection pooling with a separate connection per thread - one write connection and multiple read connections.

Ideally, all of the above would be covered by a library, and not up to the app developer.

I wrote a blog post covering some of this recently: https://www.powersync.co/blog/sqlite-optimizations-for-ultra...

Yes, that's also why I don't live in a security complex. Many people don't, but there are many people who just want that extra safety factor.

Also, the higher-end security estates are massive - you effectively have your own park (or golf course) in the estate, so that may lessen some of that. Those places are expensive, but probably still much less than a comparable house in the US or Europe.

And in cities there are places where it's safe enough to just walk around like you mentioned, but safety is something that's typically on your mind wherever you go.

But not being able to just go where I want without considering safety, and not having my kids being able to just walk around or use public transport without fearing their safety - that is a big factor in making me consider emigrating.

> Is there any sort of constantly updated "off-limits area map", or is it just common knowledge?

No maps - just common local knowledge. Some areas you avoid completely, some you just avoiding walking or driving through late at night.

> How does security in wealthier neighbourhoods look like?

Most new higher-end developments are fenced-off security complexes - electric fencing all around, and 24/7 security services. But there are still many "regular" suburbs without that. In those you typically have security patrols, and individual houses may or may not have electric fencing, depending on the area.

> Is hiking in the wild risky?

Depends on the area. In some areas you just avoid hiking alone. The more remote areas typically have no safety issues.

> Is public transport generally to be avoided, or are there "better and worse" services?

Most (but not all) wealthy people avoid local public transport ("taxis" and trains) - they do have safety (and reliability) issues. There are good options in some places - e.g. a high-end train in Johannesburg & Pretoria, and Uber is a decent option in most cities. Long distance busses and trains are also typically fine.

> As far as electricity blackouts go, everyone has a generator, right?

No, generators are mostly only by businesses. Too noisy and too much effort to use for most residences.

The cheap solutions involve getting a mini backup power supply for your internet router, and some battery-operated lights. That + some planning around e.g. cooking gets you through most outages.

Recently, battery backup + solar became very popular for everyone who can afford it. Prices for solar and batteries dropped substantially over the last 5-10 years, and the entire installed system costs around 1/3rd of what you'd pay in the US.

> Is there never a shortage of diesel for those generators?

We've had one shortage that I remember over the last couple of years. It's much more relevant for vehicle fuel than generators.

> What about the pollution that comes from running generators several hours a day?

Much less than that of cars and our coal power stations. Noise is the biggest issue.

OP here. This is a collection of app-level and library-level SQLite optimizations I've come across over the years, working with SQLite on various platforms. SQLite itself is great, but there is often a lot overhead added by higher-level libraries/bindings. This post discusses both performance optimizations ideally implemented in libraries, as well as options like JSON1 operations that could be used to make operations efficient even when the library is not.

Because many people (myself included) used .dev for internal testing domains before it was a TLD, either by specifying the domains in my hosts file, or with a local DNS server. Now i use .test for that, which is actually a reserved suffix that would never be a real TLD.

I, like many other developers, are lazy. When I search for a package, or when I want for information about a package, I just search using Google, same as when I search for anything else. No cognitive overhead to decide exactly where I should search.

Sometimes the top results for a package is its GitHub page; sometimes it's NPM. I don't particularly care which one it is, except that the NPM page very clearly shows the package name. But I do care that the results are there. And if NPM results disappeared from Google, I wouldn't remember to use NPM's search all the time.

Additionally, what from the argument does not apply to GitHub as well? Perhaps they're better at filtering out spam repositories, but otherwise it's the same thing - free hosting on a domain with presumably high ranking on Google. And if that is also removed from Google's results, NPM packages wouldn't show up anywhere in the search.

It's mentioned in the article, but the Gaggiuino project is a really nice base if you like tinkering with this kind of stuff. You start with the chassis and components of an existing machine, and just add temperature, pressure and/or flow control.

For C there should be good options.

For Rust it doesn't appear that well-supported.

A very simple approach is to poll for changes using `pg_logical_slot_get_changes()` - that should work with any driver. That's what I used for my initial experimentation, before switching over to the streaming replication protocol for better performance.

The streaming replication protocol is not that complicated, but currently you'll have to handle some of the low-level protocol yourself, or work with some very experimental implementations. There's a project to help get you started at [1], and some more discussion at [2].

For the logical decoder, wal2json is quite nice to experiment with, but I've found pgoutput is not that complicated and gives you something closer to the raw data.

[1]: https://github.com/seddonm1/logicaldecoding/

[2]: https://github.com/sfackler/rust-postgres/issues/116

Logical replication is also great for replicating to other systems - for example Debezium [1] that writes all changes to a Kafka stream.

I'm using it to develop a system to replicate data to in-app SQLite databases, via an in-between storage layer [2]. Logical replication is quite a low-level tool with many tricky cases, which can be difficult to handle when integrating with it directly.

Some examples:

1. Any value over 8KB compressed (configurable) is stored separately from the rest of the row (TOAST storage), and unchanged values included in the replicated record by default. You need to keep track of old values in the external system, or use REPLICA IDENTITY FULL (which adds a lot of overhead on the source database).

2. PostgreSQL's primary keys can be pretty-much any combination of columns, and may or may not be used as the table's replica identity, and it may change at any time. If "REPLICA IDENTITY FULL" is used, you don't even have an explicit primary key on the receiver side - the entire record is considered the identity. Or with "REPLICA IDENTITY NOTHING", there is no identity - every operation is treated as an insert. The replica identity is global per table, so if logical replication is used to replicate to multiple systems, you may not have full control over it. This means many different combinations of replica identity needs to be handled.

3. For initial sync you need to read the tables directly. It takes extra effort to make sure these are replicated in the same way as with incremental replication - for example taking into account the list of published tables, replica identity, row filters and column lists.

4. Depending on what is used for high availability, replication slots may get lost in a fail-over event, meaning you'll have to re-sync all data from scratch. This includes cases where physical or logical replication is used. The only case where this is not an issue is where the underlying block storage is replicated, which is the case in AWS RDS for example.

[1]: https://debezium.io

[2]: https://powersync.co

The article has a good explanation to quantify the security that password hashing / PBKDF2 gives, but I disagree with the conclusion. 18 bits can make the difference between a password being trivial to crack, and practically impossible to crack with current technology.

128 bits is not some magic number that needs to be achieved to have strong security.

It's printing the result of the comparison (1 or 0), not the original values.

Yes, exactly that. If it's in the white pages, everyone already has access to it.

I've seen stats that showed a significant increase in the number of callouts without major injuries in a period around 20 years ago, which correlated with mobile phones becoming popular. Suddenly it was easy for someone to call rescue services when they're stuck, where previously they would have had to make a plan. And maybe it also increased the number of people going out into the mountains now that they had an easy way to call for help.

But overall, the rescue services managed easily. The growth was slow enough over a couple of years, that it's easy enough to scale up the number of volunteers as needed. In my experience, when you get more callouts, more people are eager to volunteer. No-one wants to spend regular time doing training when there aren't any callouts.

In my current job, I have written a small framework. It's very limited in scope and hasn't changed much since the initial implementation, so it works fairly well. But, there is a big issue with getting other team members to understand how to use it. That alone is enough that we're replacing it soon.

Another team member has developed frameworks for both infrastructure management and creating services exposed over http. Here there is a large value in using the same internal tooling over 20+ projects. But once again, being an internal-only tool means that the documentation is severely lacking, and often we need new features in the frameworks. All of that is essentially blocking on the single team member. In this case we're keeping the frameworks, but it definitely has its caveats.

Update: This is an example of a compression format that allows random access to 64KB chunks, and is compatible with gzip:https://manpages.ubuntu.com/manpages/kinetic/en/man1/dictzip...

The new origin-private filesystem allows random access reads and writes. This should give the possibility to use WASM SQLite in the browser with local persistence - everything that WebSQL had.

A major advantage over WebSQL is that the developer is in complete control. The developer can pick the latest version of SQLite, enable any custom extensions (standard or additional ones), or use SQLCipher for encryption, instead of just using whatever the browser provides.

The only real disadvantage is the download size of the wasm code, but it's small enough that it won't be a blocker for sizeable interactive web apps. Just don't do this for your static blog.

SQLite is a massive library, with a lot of functionality and optional extensions, and behaviour is added or changed over versions. Codifying all the behaviour into a standard would be a massive effort. And then you're either stuck on that version of SQLite, or your standard depends on what an external project is doing with new releases - which is really not what you want for a web standard.

Alternatively, you need to pick a small stable subset of SQLite functionality to expose. But that would mean you can't just use plain SQLite anymore - you either need to modify it significantly, or need to wrap it somehow. Which either way adds a lot of overhead again.