jchanimal

The opposite. We are all learning to hone our slop detectors now, real art is more valuable and necessary.

There is a fundraiser for his family here: https://www.gofundme.com/f/honor-mikeal-rogers-by-supporting...

The word “just” in your comment disgusts me

Real innovators can’t stand this sort of noise and so it is a direct shot against their bow

The real judge will be survivorship bias and as a betting man, I might think product owners are the ones with the entrepreneurial spirit to make it to the other side.

Love your perspective. It reminds me of this argument I’m working on about Turing machine qualia. Maybe my argument is just Searle in disguise? https://x.com/jchris/status/1815379571736551923?s=46&t=8A60w...

This has always felt to me like evidence of a sort of computationalism. I am not a computationalist, but the thought is the "universal CPU" needs cycles for each particle. Mass is what takes time to process, so the voids experience no/less computational delay. This reads like the simulation author is messy and constrained, not godlike.

I came here to say the same thing. When you’re selling cloud services, the hardest thing to do is segment your customers by willingness to pay.

Cross AZ traffic is exactly the sort of thing companies with budgets need, that small projects don’t.

I had the pleasure of reading Montaigne before blogs were invented.

When I started reading blogs, the format reminded me of his essays.

I’m impressed by how easy it was, for someone new to all parts of the stack.

I agree MCP isn’t a big change, but that is a good thing. It’s more about making the implicit interface into something portable and reusable.

Most posix commands aren’t impressive in their own right, it’s the composability that gives them power. MCP plus data sync is similar.

This is just the first step in a bunch of cool stuff you can do. Now that the data is in a syncing database you can have models across multiple vendors and backends all working with the data locally while collaborating on changes. This means you can use the right model for the job without worrying about tracking where the data needs to go, instead each model can react to the appropriate fields and update types.

This is a super simple codebase so if you fork it and make something real, please share it with us.

Thanks y'all -- feedback taken. If I were saying it again I'd say something like:

Merkle proofs are rad b/c they build causal consistency into the protocol. But there are lots of ways to find agreement about the latest operation in distributed systems. I've built an engine using deterministic merge -- if anyone wants to help with lowest common ancestor algorithms it's all Apache/MIT.

While deterministic merge with an immutable storage medium is compelling, it doesn't solve the finality problem -- when is an offline peer too out-of-date to reconcile? This mirrors the transaction problem -- we all need to agree. This brings the question I'm curious about to the forefront: can a Merkle CRDT use a Calvin/Raft-like agreement protocol to provide strong finality guarantees and the ability to commit snapshots globally?

Apologies for the noise.

In founder mode, I pretty much only think about these data structures. So I am (admittedly) not that sensitive to how it comes across.

Spam would be raising the topic on unrelated posts. This is a context where I can find people who get it. The biggest single thing we need now is critical feedback on the tech from folks who understand the area. You’re right I probably should have raised the questions about mergability and finality without referencing other discussions.

Because I don’t want to spam, I didn’t link externally, just to conversation on HN. As a reader I often follow links like this because I’m here to learn about new projects and where the people who make them think they’ll be useful.

ps I emailed the address in your profile, I have a feeling you are right about something here and I want to explore.

I figured the responses would be more interesting. Questions about CRDT guarantees etc.

Perhaps worth seeding the convo with a remark about finality.

We launched Fireproof earlier this month on HN. It’s a tamperproof Merkle CRDT in TypeScript, with an object storage backend for portability.

See our Show HN: https://news.ycombinator.com/item?id=42184362

We’ve seen interest from trading groups for edge collaboration, so multi-user apps can run on-site without cloud latency.

Thank you! I'm excited to see more and more people try it out as it gets more stable and easy to use.

Thanks, and thanks for the encouragement to fully document the gateway interface. We have been flux-ing it lately but as soon as it settles down we’ll do that.

The vision is many small ledgers, so the full replication per ledger makes sense, but we have work to do on cross-ledger queries

We focus on provable data integrity and being easy enough that new devs can find success without learning much stack.

Many alternatives start with a backend integration mindset, I started from the React hooks and worked towards the storage engine.

I answered 1-5 in my other reply, hit save and so here's more:

6. There is a limit parameter on the query API, and the underlying data structures utilize async iterator patterns so we have a go-forward path to a streaming (data / query larger than memory) implementation. But for now the decrypt implementation is eager instead of lazy, so that's the first place we'd want to focus to make data > memory workloads no problem.

7. Other embedded databases don't have auth, but we are network-aware so it's a different ballgame. Our next step is read/write access control on a per-ledger basis.

UCAN capability delegation allows us to keep an embedded mindset here, in that authorization becomes a matter of data validity, not something that has to be fetched from a centralized resources. How it works: client device agents generate non-extractable keypairs (like Passkeys) and can link them to account principals via any signing endpoint Fireproof trusts (for starters just the one we run, to an end user it looks like clicking a validation link in an email.) Agents create a new cloud database clock register by locally generating an ephemeral keypair that signs itself over to the principal. Our centralized clock register endpoint only allows updates to the resource identified by the clock's public key ID, from agents which have a valid signed delegation chain to the ephemeral key.

To a developer it will look something like `db.share("bob@example.com")` and now Fireproof Cloud will let Bob read and/or write the db also.

What's cool about this is that access control changes are just data manipulations, so they can happen offline. And the valid delegations can be safely delivered over any channel. In fact there are no secrets in this system except for the non-extractable keypairs.

If you are thinking to yourself "what about revocation?" -- we are hiring.

8. The sync endpoint has the minimal blob k/v (no list) and register APIs. And can all be floated on top of any raw kv with check-and-set semantics if needed.

We have plans for a REST API in Fireproof Cloud, where if you allow the cloud to decrypt and process your data, we can give you raw queries instead of you replicating and then querying locally. I am thinking a CSV output here would be a good place to start.

9. Runs great anywhere JS runs. We have examples (like CatBot linked above) that subscribe to the ledger on the backend and operate locally, often responding the user events. So the DB is acting as an RPC bus... this is a common pattern in CouchDB so I made sure Fireproof works great like that.

To run in an edge function, you usually aren't gonna replicate to local filesystem, instead you can configure the database to read and write directly with the cloud store. Because of the eager decrypt we do, this is actually pretty fast and not that chatty.

10. The CodePen demo on our homepage is an iframe, works great. We have a contributor (I think I see in the thread here) who is working on React Native -- most of the heavy lift is done, but our gateway interface is only now settling down to where it makes sense to finalize the integration. I have also done Socket Supply for mobile and that works great.

These are awesome questions, I'll try to fold the answers into the docs also.

1. The embedded database subscribes to the remote sync endpoint when it is connected. This subscription might be polling, websocket, or anything else. The local embedded database will try to keep up with changes anyone pushes to the remote endpoint. This is more a backend mechanical thing than an API you'll see.

Your code can subscribe to the local database -- this is a JavaScript event loop, and any updates, local or remote, will cause your callback to run. The upshot is all you have to do is connect your database to the sync endpoint and it will stay up to date, and you can also connect your UI to the database via `db.subscribe()`

2. Updates are written to local storage (indexed db or the filesystem) as encrypted blobs. These are then replicated to the cloud (without being parsed by the cloud). We have SQL connectors also, but we haven't done the Postgres specific stuff (just started designing it). That is the data side. There is also the clock register, which the client updates to point to the most recent blob. This register is multi-writer safe, and can occasionally point to more than one "head" blob, in which case the client does the deterministic merge on read.

3. In my experience most people use the defaults, so we have Fireproof Cloud which uses R2 and durable objects. We also have a SAM template for AWS, and a connector for Netlify, in addition things that are more like parts for building your own backend (file and http endpoints).

4. Each ledger replicates 100% when it syncs, so all hosts have the same data (no sharding within a ledger.) Typically you have one centralized endpoint to sync via. (p2p is possible but you'd end up contributing some plumbing to the project I bet). So in this case the class would have a URL that is the sync point, and everyone would pull from it periodically or via streaming.

Merges are idempotent, deterministic, associative, and commutative, so it doesn't matter what order the teacher and students apply updates to their local instance, once all updates are applied, they have the same state.

5. The e2e encryption means you'd have to give the keys to the server to allow it to create subsets for sync, so we haven't done that yet. Our next optimization is to sync the readonly current dataset first, then any extra data needed for writing, and only when necessary, the historical log. This still doesn't solve the subset sync issue, but will benefit all use cases immediately.

There is some cool research we might use for subset sync: https://g-trees.github.io/g_trees/

But more practical is probably to finish the Postgres backend and then build subsetting at the global (multi-ledger) dataset level.

The default gateway style doesn’t have to parse the encrypted files so the endpoint could be anything like an S3 bucket and anything like a register to redirect to the most recent file.

Someone has a similar question in our discord, where I described what we’re planning with Postgres and unencrypted sync.

Here's Mykle's project https://msl.mykle.com/2024/10/bloopernet-fp-a-serverless-web...

We'll be doing a drum machine bike ride in Atlanta in December. Ping me on discord if you want to join.

I wrote a "smart book" project with vector search in the browser with Fireproof. Here is the vector integration: https://github.com/jchris/smart-book/blob/e2ab5d3ee94846e6ac...

We are working on transparent replication to Postgres JSON which will make it a lot easier to adopt.

There are maybe too many backend implementions: https://github.com/fireproof-storage/connect/tree/main/src

And writing a custom one is super-easy (docs coming soon, for now here is the interface): https://github.com/fireproof-storage/fireproof/blob/c4462c8a...

Thanks for the feedback. That is a bridge we are crossing now. We started with open source backends for maximum coverage, and are standing up Fireproof Cloud now as the easy default. If you click "inspect data" in the demo CodePen app, it will show you a live replication in our dashboard.

We'll be adding auth and sharing controls soon, so you should be able to build basic apps with just Fireproof and an HTML host.

Thanks for pointing out the key management potential!

Thanks and thanks for the early feedback last year when I was still working on the alpha version.

This "explains the launch of the Chicken Big Mac from an economic point of view" -- this is like the Asymco of fast food.

I think the front end is the most interesting place right now, because it’s where people are making stuff for themselves with the help of LLMs.

The browser is a great place to build voice chat, 3d, almost any other experience. I expect a renewed interest in granting fuller capabilities to the web, especially background processing and network access.