HNHacker News
TopNewBestAskShowJobs

matharmin

1,706 karma · joined October 2, 2015

Co-founder at JourneyApps, currently focusing on PowerSync.

https://powersync.com

submissionscomments
matharmin··on Hosting SQLite databases on any static file hoster (2021)
I think another issue with compressing partial content is that those compressed responses cannot be cached efficiently - it would have to be compressed on-the-fly for every range requested. And while compression is not as computationally expensive as it used to be, it does still add overhead that could be more than the bandwidth overhead of uncompressed data.

There should be a workaround for this using a custom pre-compression scheme, instead of relying on HTTP for the compression. Blocks within the file will have to be compressed separately, and you'll need some kind of index mapping uncompressed block offsets to compressed block offsets.

Unfortunately there doesn't seem to be a common of-the-shelf compression format that does this, and it means that you can't just use a standard SQLite file anymore. But it's definitely possible.

matharmin··on Google officially launches Equiano subsea cable
While latency to the rest of the world is high, the connectivity is actually pretty decent over the last couple of years. For around 50 USD per month I get a 100Mbps residential fibre connection, and I get close to the maximum speed to the US or Europe.

That is not quite the same things as the dedicated connections that AWS/Google would want between their data centres, but it's not the same connectivity issues we struggled with 15 years ago when we only had one or two subsea cables.

matharmin··on Matter – Protocol to connect compatible devices and systems with one another
Thanks, this is the first explanation I've seen of how the ecosystem actually fits together, and what Thread and Matter would mean for end users.
matharmin··on Flyweight: An ORM for SQLite
With SQLite, I've started using json_each on a JSON parameter for bulk inserts or updates. Other SQL databases should have something similar.

It's much cleaner than generating SQL, and doesn't run into issues with exceeding the maximum number of parameters.

matharmin··on Ignite – Use Firecracker VMs with Docker images
Docker for Windows/Mac uses a single VM for all your containers. Firecracker/Ignite has a dedicated micro-VM for each container - a completely different architecture, with much better isolation between containers. And the 125ms boot time is not something you get with a typical VM.

But I do agree that they don't explain how Ignite is different from "wrapping a container in a VM layer" that Kata/gVisor does.

matharmin··on DJI Drone Flies over the Top of Mount Everest
I don't have any specific stories, but my local rescue team in Cape Town now uses a drone on many mountain rescue callouts to assist in locating the patients.

It is surprisingly difficult to use a drone effectively for that - it is much harder to spot someone from a drone's camera than you'd expect. It is often still faster to find the patients by foot, but it has helped in some cases. And when the patients have been found, the drone can help to scout the surrounding area to find the best route out.

So overall the drone has been some help, but it hasn't significantly affected the outcome of rescue operations in any case that I know of.

matharmin··on Programming languages endorsed for server-side use at Meta
Where's the argument? It makes broad claims about the languages, that many people would disagree with, with no substantiation.
matharmin··on Magnus Carlsen to give up World Championship title
There is not that much overlap between skills required for chess and StarCraft. Being good at one may perhaps correlate with having some natural talent in the other, but experience in the specific game would matter a lot more.
matharmin··on Copilot sells code other people wrote
For public repositories, whether copying small parts of code is considered fair use is just a copyright question.

On the other hand, if you copy from private repositories, it quickly gets into the territory of stealing trade secrets.

matharmin··on Ruby YJIT Ported to Rust
If you want to design your own CPU, supporting LLVM is going to give you much greater benefits than supporting Ruby. Nevermind the fact that you don't even need this to support Ruby.
matharmin··on Frawk: A fast, JITted, statically-typed AWK written in Rust
This is a different compiler/interpreter for the same language - nothing prevents it from being a drop-in replacement.
matharmin··on Ask HN: Are “no-code” and “serverless” terms misleading?
A closer analogy would be renting a car. And if you rent a car full-time, you can't really call yourself car-less.

In both cases there is a lot of grey areas, but the general concept is the same: If you don't own, maintain or operate (drive) a server/car, you can call yourself server/car-less, regardless of whether a server or car is still being used.

matharmin··on Reimagining Chess with AlphaZero
From the posted stats, the Stalemate=win rule does result in more decisive games (fewer draws). But it has an even bigger effect in giving white a further advantage over black.
matharmin··on Tell HN: AWS appears to be down again
In this case, this was specifically with a single-AZ setup, using an AZ that was supposed to be unaffected.
matharmin··on Pre-commit: framework for managing/maintaining multi-language pre-commit hooks
IMO linters definitely belong into the CI/CD pipeline. I do block a release if there are whitespace issues - it's trivial for the developer to fix, and part of basic code hygiene standards that we enforce in every repository.

And if there is an emergency hotfix that needs to go out as soon as possible, we do have processes in place to bypass the CI/CD checks.

matharmin··on Tell HN: AWS appears to be down again
Yes, I've seen issues that affected the entire region. In my specific case, I happened to have an ElastiCache cluster in the affected AZ that became unreachable (my fault for single AZ). But even now, I'm unable to create any new ElastiCache clusters in different AZs (which I wanted to use for manual failover). And there were a lot of errors on the AWS console during the outage.

"almost unusable" is maybe exaggerating, but there were definitely issues affecting more than just the single AZ.

matharmin··on Tell HN: AWS appears to be down again
AWS is under-reporting the severity of the issue though. The primary outage may be in a single AZ, but there are parts of the AWS stack that affected all AZs in us-east-1, and potentially other regions as well. For example, even now I'm unable to create a new ElastiCache cluster in different AZs of us-east-1.
matharmin··on Pre-commit: framework for managing/maintaining multi-language pre-commit hooks
I like seeing exactly what I commit, so I never let a pre-commit script make any changes. It's just a single command to format any changed files in the repo.

But some developers in my team do like that convenience of just being able to commit, so that's an option that can be configured in our internal tooling.

matharmin··on Pre-commit: framework for managing/maintaining multi-language pre-commit hooks
The CI check is the important one: It's what ensures that bad formatting doesn't make it into master.

The pre-commit is just a convenience for the developer. It gives faster feedback - immediately when trying to commit, instead of a couple of minutes later. And if a developer on my team wants to disable it, it doesn't affect others.

And yes, it's important that you use the same version in dev and on CI/CD. The same applies for practically any other tool you use in your project. I see the CI/CD config as the "master" version - it's a known-working setup, that any developer should be able to use.

matharmin··on We check our node_modules folder into source control
The pnp.js file has a list of modules, not the modules themselves. The modules are still read from zip files at runtime.

Yarn doesn't do any kind of compilation itself - that would be done by build tools like webpack.

matharmin··on Amazon employees in 20 countries will strike on Black Friday
They can be both, but in many of the listed countries there aren't really Amazon workers (at least not warehouse staff), and they are not "[striking] for better working conditions" as stated in the headline.

That is part of the #MakeAmazonPay movement, but it's not all the movement is, and saying that employees are striking in 20 countries is completely false.

matharmin··on The Toit language is now open source
I'd assume offline features would keep functioning, but online features would stop working. The online features seem to be device provisioning (so may not be able to change WiFi access points?), OTA updates, and cloud PubSub.
matharmin··on Amazon employees in 20 countries will strike on Black Friday
Some of those countries don't even have any Amazon distribution centres. A lot of those are protests by activists - not workers, and not related to working conditions at all. The title is not just misleading - it's completely wrong.
matharmin··on Reproducible builds for Debian: a big step forward
Yes, it's a one-way function. And if the builds are reproducible, this function is deterministic.

The way to verify is to redo the entire compile process. But the advantage is that anyone can do this, so you can have other users in the community validating the builds.

matharmin··on QR error correction helps and hinders scanning
Another interesting effect: If you have a version 1 code (the smallest size), there is no "alignment" block in the lower-right corner - only the main three positional blocks. Some scanning algorithms (such as the popular zxing library) struggle with this, and it actually works better to use a version 2 code.
matharmin··on Show HN: FractalCrypt 2.0 – free deniable encryption cryptoarchiver
There is no way for the attacker to know how many keys you have. So you can give the attacker 2 keys, while you have your actual sensitive data behind the 5th one.

It could still be a challenge to convince the attacker that you really only had n-1 keys, so you may need to include plausibly-sensitive data in earlier layers.

matharmin··on A future for SQL on the web
SQLite as a library is amazing, SQLite as a spec for a standard web API is not.

The issue is not just that there wasn't a competing implementation at the time, it is that you could not feasibly create a competing implementation. The set of features supported by SQLite is _massive_. And even you stick with SQLite, which version? And which extensions? Which features do you remove because they're not safe or not applicable for web?

Alternatively you could start from the ground up and create a new database spec that is similar to SQLite, but more limited in scope. But then it will not be directly compatible with SQLite, and need a translation layer. The browser would likely have to re-implement large parts of SQLite for this to work. And you'd miss out on so much of the functionality that makes SQLite attractive to use.

Arguably that would still have been better than IndexedDB. All that IndexedDB has going for it, is that it is simple to implement. But even then Safari manages to get the implementation horribly broken so often.

What we need instead is to have a low-level storage api that can be used to implement other databases as libraries, including SQLite. The Storage Foundation API, mentioned in the post, might just give us that.

matharmin··on Google results for PHP tutorials contain SQL injection vulnerabilities
Even better, you can pass it in a JSON array, and use the built-in JSON functionality to iterate through it. I started doing this in SQLite, and it made some queries a lot cleaner.
matharmin··on Two Wishes for Dev Tooling
I strongly disagree with "bot refactors". Automated or semi-automated refactoring is great, but it's best run offline on a developer's machine.

Even if the system to do that works perfectly, you've still added more noise to the commit history, and in my opinion it still doesn't have any real advantages over doing it locally.

Our system for formatting code (actually similar to the author's current system):

1. The developer can choose whether or not to do auto-format on save.

2. The developer can run a single command in the repo to format all staged files. (Formatting all files in the repo is too slow).

3. A pre-commit hook checks that all changed files are correctly formatted.

4. On CI we check that all files are correctly formatted (in case any of the above didn't happen on the developer's machine).

It takes some effort to setup the system initially, but from then on it works really well. You get these advantages:

1. Formatting (or making any other change to a file) is always an explicit action.

2. The developer can review all changes before committing.

3. Every step of the process is easy to debug if something goes wrong (which is very rare).

4. There is minimal overhead to applying the auto-formatting.

5. Developers still have freedom in their personal workflows.

One example I've seen of a bot automatically creating PRs is dependabot to automatically update dependencies, but in my opinion it adds way more noise than value. Running a command locally to do the updates is just as easy, without any of the noise.

matharmin··on IndexedDB is completely broken in latest Safari
SQLite is amazing. But after having worked with it for many years now, I believe it should never be a platform-level API, and not standardizing it for browsers was the right choice.

SQLite has a _lot_ of great features. Many of them only introduced in recent versions, and many of them are optional extensions enabled at compile-time. There is no feasible way to standardize this to get the same functionality across all browsers.

IndexedDB may not be great to work with (at least not without helper libraries), but at least the API surface is small, and much easier to ensure you get the same behaviour in all browsers.

I'd rather advocate for a proper block-level filesystem API that could be used as backing storage for a WASM SQLite library.

← PreviousPage 6 of 11Next →