But to be fair, we're not a company, but a working group. And we don't hate hypertext, we just haven't gotten around to implementing server-side rendering, which would be a nice thing to do.
3,057 karma · joined May 5, 2012
https://invisible.college/@toomim
https://braid.org
But to be fair, we're not a company, but a working group. And we don't hate hypertext, we just haven't gotten around to implementing server-side rendering, which would be a nice thing to do.
It's important for this to be optional. Consider that each keystroke that edits a text resource changes its version ID. You want to be able to link to the text without changing the ID with each keystroke.
It's also important to be able to pass versions within headers of a request or response. The server wants to update the version ID with each update it sends to the client, and that's most elegantly done in a header.
As for your skepticism about integrating synchronization into HTTP instead of embedding it within a datatype, I can empathize -- but you might be surprised at just how elegantly HTTP extends into a full-featured synchronization protocol. A key to this elegance is the Merge-Type: this is the abstraction that allows a single synchronization algorithm to merge across multiple data types.
As an application programmer, you will specify both the data types of your variables (e.g. int, string, bool) and also the merge-types (e.g. "this merges as a bank account balance, or a LWW unique ID, or a collaborative text field"). This is all the application programmer needs to specify. The rest of the synchronization algorithm gets automated by middleware libraries that the programmer can just use and rely upon, like his compiler, and web browser.
I'd encourage you to check out the Braid spec, and notice how much we can do with how little. This is because HTTP already has almost everything we need. Compare this with the WebDAV spec, for instance, which tries to define versioning on top of HTTP, and you'll see how monstrous the result becomes. Example here: https://news.ycombinator.com/item?id=40481003
> The focus is to help application developers to not get into the shenanigans of CRDTs and distributed algorithms etc., but solve those things already via polyfills.
Yes, this ^^ is the initial focus, and is enough to provide value now.
However, ultimately we will generalize HTTP into a fully peer-to-peer system. Each extension we add provides a new dimension of p2p, and at some point we won't need servers at all. The big blocker to that, right now, is TLS+DNS, which are baked into the definition of https:// URIs. More on this at https://braid.org/meeting-2 in the "HTTP2P" talk.
Updates (as requested by d-z-m):
- Link to talk in Meeting 2: https://braid.org/video/https://invisiblecollege.s3.us-west-...
- Slides: https://braid.org/files/http2p2p.pdf
- The specific section on TLS+DNS: https://braid.org/video/https://invisiblecollege.s3.us-west-...
SSE doesn't work because it (1) doesn't support binary data, (2) assumes that versioning is linear (which doesn't work in a distributed system), and doesn't provide any way to add new headers to each response.
SSE also has an awkward encoding format, but that's neither here nor there.
HTTP is a standard on top of TCP. It provides a higher-level abstraction than a socket -- the abstraction of State Transfer. When you use a WebSocket, you're back to a low-level socket, and have to redefine "state", and the methods to get it, put it, and subscribe to it.
Since each web programmer defines those methods in a different way, his state gets hidden behind his own non-standard protocol. There is no way for website A to re-use the state on website B, for instance, without learning website B's custom WebSocket protocol and re-implementing it perfectly.
CDNs and other caches cannot handle WebSocket traffic. But if you use a standard like Braid-HTTP, they can cache your dynamic assets along with your static assets.
This algorithm is the first to do so, and happened to be developed in the Braid group.
As for scaling, check out JosephG's https://josephg.com/blog/crdts-go-brrr/ CRDT, which scales great, and is thoroughly tested. The Braid protocol is also architected with a new type of OT/CRDT architecture called a Time Machine that lets you do advanced things like apply backpressure through a network to decrease the frequency of updates, which we presented in https://braid.org/meeting-81, and are releasing in the https://github.com/braid-org/braid-text library that you can try right now.
Some people have suggested using chunked boundaries as a way to encode multiple responses to a subscription. Perhaps this is what you are thinking of. However, chunk boundaries cannot be relied upon to remain intact; they can be changed by intermediaries. The chunks are only an encoding of the transfer. They are not supposed to contain any semantic information.
Chunked transfer is also disallowed in http/2.
Let's say that your client runs GET /some-json, and that JSON gets updated, and the client wants to get the updates. Right now, your options are:
1. Re-run the GET /some-json again from the client (polling)
2. Start a websocket, and invent some custom protocol over the websocket to let the client subscribe to /some-json.
With Braid, you just do:
GET /some-json
Subscribe: true
And the client will automatically get the updates, within HTTP. The Braid-HTTP ponyfill library abstracts the details behind a normal fetch() for you, until browsers implement it.Today, we tend to use HTTP for static assets, and then switch to a websocket or SSE or something whenever things get dynamic. That sucks! We lose the standard of HTTP!
Braid lets you keep using HTTP, even for your dynamic assets. It also solves issues in caching. Instead of using max-age heuristics, we have actual real versioning!
And yes— the protocol generalizes all the way to full-peer-to-peer multi-writer CRDT/OT algorithms. But start simple!
Braid is transitioning HTTP into a peer-to-peer world, where the distinction between clients and servers doesn't matter.
HTTP was invented for static pages, that were written by hand, and rarely changed. But today's web has dynamic pages, driven by javascript and databases, and users expect them to update in realtime. Braid-HTTP adds support for dynamic state to HTTP.
The HTTP acronym stands for HyperText Transfer Protocol. ReST stands for Representational State Transfer.
HTTP began as a HTML Transfer Protocol, but then generalized to content-types beyond HTML, such as images, scripts -- and general state. So today HTTP is known as a State Transfer Protocol. This architecture was canonicalized as ReST in Roy Fielding's dissertation.
In order to generalize HTTP from State Transfer to State Synchronization, we have to augment some parts of it, such as request/response. Instead of just getting a single response to a request, Braid allows a client to subscribe to a resource, with a single request which will be then given multiple responses -- one response for each update to the resource.
It is quite elegant to extend HTTP at this level! And we can do so without requiring any changes to web browsers.
https://github.com/braid-org/braid-text/
https://www.npmjs.com/package/braid-text
This is the easiest way to add collaborative editing to a web app. You can add it to any (req, res) handler in your nodejs app. No websocket necessary, since it extends HTTP itself!Plus, it features a new `simpleton` merge-type that requires zero history overhead on the client, and you can implement the client protocol from scratch in just 50 lines of code. There's no reason not to add collaborative editing to every user-editable string in your app now!
All the network traffic is just HTTP, with an open, easy-to-read protocol that supports any OT or CRDT (currently defaulting to josephg's Diamond-Types). You can even read and write directly using the Braid-Chrome extension, which adds a devtools panel to view the version history of any Braid-HTTP resource: https://github.com/braid-org/braid-chrome.
I'm really excited about this library. We were going to announce it tomorrow (at https://braid.org/meeting-86), but since this hacker news discussion is happening... I can't hold myself back from talking about it now!
Let's compare WebDAV with Braid for something simple like looking up the current version of the resource.
In WebDAV, you have to:
- Send a PROPFIND request to the URL of the resource you want to check. In the request body, specify the DAV:version-controlled-binding property you want to retrieve, using a custom XML syntax. Here's an example:
PROPFIND /path/to/resource HTTP/1.1
Host: example.com
Depth: 0
Content-Type: application/xml
<?xml version="1.0" encoding="utf-8"?>
<propfind xmlns="DAV:">
<prop>
<version-controlled-binding />
</prop>
</propfind>
- The server responds with a 207 Multi-Status response, with XML to parse: HTTP/1.1 207 Multi-Status
Content-Type: application/xml
<?xml version="1.0" encoding="utf-8"?>
<multistatus xmlns="DAV:">
<response>
<href>/path/to/resource</href>
<propstat>
<prop>
<version-controlled-binding>
<href>/path/to/version/history</href>
</version-controlled-binding>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
</multistatus>
This does not yet give you the version -- it tells you a separate URL that contains the version history. So now you have to query that for the version:- Send a REPORT request to that url, and in the request body, specify the DAV:version-tree report you want to retrieve:
REPORT /path/to/version/history HTTP/1.1
Host: example.com
Content-Type: application/xml
<?xml version="1.0" encoding="utf-8"?>
<version-tree xmlns="DAV:">
<prop>
<version-name />
<creator-displayname />
<creation-date />
<comment />
</prop>
</version-tree>
- The server responds with a 207 Multi-Status, with more custom XML for you to parse: HTTP/1.1 207 Multi-Status
Content-Type: application/xml
<?xml version="1.0" encoding="utf-8"?>
<multistatus xmlns="DAV:">
<response>
<href>/path/to/version/1</href>
<propstat>
<prop>
<version-name>1.0</version-name>
<creator-displayname>John Doe</creator-displayname>
<creation-date>2023-05-26T10:00:00Z</creation-date>
<comment>Initial version</comment>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
<response>
<href>/path/to/version/2</href>
<propstat>
<prop>
<version-name>2.0</version-name>
<creator-displayname>Jane Smith</creator-displayname>
<creation-date>2023-05-27T14:30:00Z</creation-date>
<comment>Updated version</comment>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
</multistatus>
You can find the versions in there, and if you then parse and filter to the most recent version, you can see that the current version is 2.0.That was a LOT of work!
Now let's see it in Braid.
To get the current version, just do a normal GET or a HEAD, and look at the Version: header in the response:
Request:
GET /path/to/resource HTTP/1.1
Response: HTTP 200 Ok
Version: "2.0"
<body of resource>
That's it! It's that simple!Braid is this simple because it is baked into HTTP. It turns out that HTTP itself is already very close to a State Synchronization protocol -- it just needs a few headers, and the ability to stream updated responses when a resource changes. I encourage you to check out the spec, and see how easy it is to do things that were complicated or impossible in WebDAV:
https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-b...
Braid proposes new HTTP headers that give HTTP new features, so that instead of just being a State Transfer protocol, it gains the functionality to become a full State Synchronization protocol.
To "extend" a protocol means to add new features to it, e.g. with new headers, which doesn't break existing uses of the protocol, but allows new implementations to opt into new extended functionality.
HTTP was originally stateless, but then added cookies. Over time, web apps evolved to store tons of state in databases on the server. Then they started storing tons of state in javascript variables and DOM on the client. Today, most of the code we write in a web app is there to synchronize the changing state on a server with the changing DOM on the client. The web is no longer static pages. It is dynamic state. But since we haven't extended HTTP to support dynamic state, programmers have to write all this synchronization code by hand. This has been a pain in the ass, and so we've evolved monstrous stacks of Javascript frameworks (react, redux, etc. etc.) to try to help manage this state synchronization from server to client. If we build this functionality directly into HTTP, tons of code goes away, and the web itself gains great new features. Then, since we will have a standard for state synchronization, our state can interoperate, and walled siloes of websites break down. We can have a P2P web.
When this work is complete, the browser will no longer need a reload button. The browser/server will guarantee that every page is always up-to-date, and every peer that synchronizes with that state will have an equal copy.
Most of the implementations are in Javascript. Check out braid-text, it's super practical: https://www.npmjs.com/package/braid-text
There are also implementations in Rust and Go.
It also allows standard tools and front-ends to work with any back-end. We could have CDNs that can cache and sync dynamic HTML for a set of users without touching the server.
I'm really curious what the cost per byte would be to make it worthwhile to offer a "host this byte forever, for one up-front fee" service.
I happen to be working on a standardized protocol (braid-http) for sending diffs over HTTP, and have been thinking about applying it to hyperdiv. I'd love to understand your use-case and what about plain HTML you like here.
It's so funny how subjective that is — I liked the audio enough to tell the author about it in a DM. Then I come to HN and read that someone disagrees. :)
If you live in an area of low crime, though, maybe it'll be fine not to answer phone calls from numbers that aren't in your phone.
Rent control incentivizes empty housing during bear markets.