HNHacker News
TopNewBestAskShowJobs

toomim

3,057 karma · joined May 5, 2012

Independent Scientist in Computer Science and Psychology

https://invisible.college/@toomim

https://braid.org

submissionscomments
toomim··on Braid: Synchronization for HTTP
Point taken.

But to be fair, we're not a company, but a working group. And we don't hate hypertext, we just haven't gotten around to implementing server-side rendering, which would be a nice thing to do.

toomim··on Braid: Synchronization for HTTP
Yes, we will want to extend URLs to standardize an optional version identifier.

It's important for this to be optional. Consider that each keystroke that edits a text resource changes its version ID. You want to be able to link to the text without changing the ID with each keystroke.

It's also important to be able to pass versions within headers of a request or response. The server wants to update the version ID with each update it sends to the client, and that's most elegantly done in a header.

As for your skepticism about integrating synchronization into HTTP instead of embedding it within a datatype, I can empathize -- but you might be surprised at just how elegantly HTTP extends into a full-featured synchronization protocol. A key to this elegance is the Merge-Type: this is the abstraction that allows a single synchronization algorithm to merge across multiple data types.

As an application programmer, you will specify both the data types of your variables (e.g. int, string, bool) and also the merge-types (e.g. "this merges as a bank account balance, or a LWW unique ID, or a collaborative text field"). This is all the application programmer needs to specify. The rest of the synchronization algorithm gets automated by middleware libraries that the programmer can just use and rely upon, like his compiler, and web browser.

I'd encourage you to check out the Braid spec, and notice how much we can do with how little. This is because HTTP already has almost everything we need. Compare this with the WebDAV spec, for instance, which tries to define versioning on top of HTTP, and you'll see how monstrous the result becomes. Example here: https://news.ycombinator.com/item?id=40481003

toomim··on Braid: Synchronization for HTTP
> Why "peer-to-peer"? While Braid can be used in a local-first setting and probably do peer-to-peer, that isn't the focus here.

> The focus is to help application developers to not get into the shenanigans of CRDTs and distributed algorithms etc., but solve those things already via polyfills.

Yes, this ^^ is the initial focus, and is enough to provide value now.

However, ultimately we will generalize HTTP into a fully peer-to-peer system. Each extension we add provides a new dimension of p2p, and at some point we won't need servers at all. The big blocker to that, right now, is TLS+DNS, which are baked into the definition of https:// URIs. More on this at https://braid.org/meeting-2 in the "HTTP2P" talk.

Updates (as requested by d-z-m):

- Link to talk in Meeting 2: https://braid.org/video/https://invisiblecollege.s3.us-west-...

- Slides: https://braid.org/files/http2p2p.pdf

- The specific section on TLS+DNS: https://braid.org/video/https://invisiblecollege.s3.us-west-...

toomim··on Braid: Synchronization for HTTP
It works just fine. Existing servers, proxies, and middleboxes just pass through the response if they don't understand status 209. The braid.org site itself is running on Braid-HTTP, through a reverse proxy, and everything is peachy keen.

SSE doesn't work because it (1) doesn't support binary data, (2) assumes that versioning is linear (which doesn't work in a distributed system), and doesn't provide any way to add new headers to each response.

SSE also has an awkward encoding format, but that's neither here nor there.

toomim··on Braid: Synchronization for HTTP
WebSockets and SSE are standards in the same way that TCP is a standard -- if you use them, you are still defining an ad-hoc protocol on top to subscribe to state and publish new state.

HTTP is a standard on top of TCP. It provides a higher-level abstraction than a socket -- the abstraction of State Transfer. When you use a WebSocket, you're back to a low-level socket, and have to redefine "state", and the methods to get it, put it, and subscribe to it.

Since each web programmer defines those methods in a different way, his state gets hidden behind his own non-standard protocol. There is no way for website A to re-use the state on website B, for instance, without learning website B's custom WebSocket protocol and re-implementing it perfectly.

CDNs and other caches cannot handle WebSocket traffic. But if you use a standard like Braid-HTTP, they can cache your dynamic assets along with your static assets.

toomim··on Braid: Synchronization for HTTP
You can use CRDT algorithms with Braid that heal from network partitions transparently, and guarantee strong eventual consistency. Here's an example CRDT algorithm for collaborative text that can handle arbitrary network partitions, and guarantee full eventual consistency, and prune all unnecessary history:

https://braid.org/antimatter

This algorithm is the first to do so, and happened to be developed in the Braid group.

As for scaling, check out JosephG's https://josephg.com/blog/crdts-go-brrr/ CRDT, which scales great, and is thoroughly tested. The Braid protocol is also architected with a new type of OT/CRDT architecture called a Time Machine that lets you do advanced things like apply backpressure through a network to decrease the frequency of updates, which we presented in https://braid.org/meeting-81, and are releasing in the https://github.com/braid-org/braid-text library that you can try right now.

toomim··on Braid: Synchronization for HTTP
Transfer-Encoding: chunked is for something completely different. It is for streaming a single body that you do not know the length of. It does not define a subscription format. It does not specify versioning. It does not define a patch format.

Some people have suggested using chunked boundaries as a way to encode multiple responses to a subscription. Perhaps this is what you are thinking of. However, chunk boundaries cannot be relied upon to remain intact; they can be changed by intermediaries. The chunks are only an encoding of the transfer. They are not supposed to contain any semantic information.

Chunked transfer is also disallowed in http/2.

toomim··on Braid: Synchronization for HTTP
Yes, or even more simply:

Let's say that your client runs GET /some-json, and that JSON gets updated, and the client wants to get the updates. Right now, your options are:

1. Re-run the GET /some-json again from the client (polling)

2. Start a websocket, and invent some custom protocol over the websocket to let the client subscribe to /some-json.

With Braid, you just do:

    GET /some-json
    Subscribe: true
And the client will automatically get the updates, within HTTP. The Braid-HTTP ponyfill library abstracts the details behind a normal fetch() for you, until browsers implement it.

Today, we tend to use HTTP for static assets, and then switch to a websocket or SSE or something whenever things get dynamic. That sucks! We lose the standard of HTTP!

Braid lets you keep using HTTP, even for your dynamic assets. It also solves issues in caching. Instead of using max-age heuristics, we have actual real versioning!

And yes— the protocol generalizes all the way to full-peer-to-peer multi-writer CRDT/OT algorithms. But start simple!

toomim··on Braid: Synchronization for HTTP
That is cool! Nobody has worked on a FireFox devtool yet, but we welcome contributions.
toomim··on Braid: Synchronization for HTTP
You could sync two servers. Or you could sync a server with clients.

Braid is transitioning HTTP into a peer-to-peer world, where the distinction between clients and servers doesn't matter.

toomim··on Braid: Synchronization for HTTP
Yes, and to elaborate — for any HTTP usage with dynamic state.

HTTP was invented for static pages, that were written by hand, and rarely changed. But today's web has dynamic pages, driven by javascript and databases, and users expect them to update in realtime. Braid-HTTP adds support for dynamic state to HTTP.

toomim··on Braid: Synchronization for HTTP
> HTTP ... doesn't deal with state transfer nor does it define it.

The HTTP acronym stands for HyperText Transfer Protocol. ReST stands for Representational State Transfer.

HTTP began as a HTML Transfer Protocol, but then generalized to content-types beyond HTML, such as images, scripts -- and general state. So today HTTP is known as a State Transfer Protocol. This architecture was canonicalized as ReST in Roy Fielding's dissertation.

In order to generalize HTTP from State Transfer to State Synchronization, we have to augment some parts of it, such as request/response. Instead of just getting a single response to a request, Braid allows a client to subscribe to a resource, with a single request which will be then given multiple responses -- one response for each update to the resource.

It is quite elegant to extend HTTP at this level! And we can do so without requiring any changes to web browsers.

toomim··on Braid: Synchronization for HTTP
We're about to release a new braid-text library:

    https://github.com/braid-org/braid-text/
    https://www.npmjs.com/package/braid-text
This is the easiest way to add collaborative editing to a web app. You can add it to any (req, res) handler in your nodejs app. No websocket necessary, since it extends HTTP itself!

Plus, it features a new `simpleton` merge-type that requires zero history overhead on the client, and you can implement the client protocol from scratch in just 50 lines of code. There's no reason not to add collaborative editing to every user-editable string in your app now!

All the network traffic is just HTTP, with an open, easy-to-read protocol that supports any OT or CRDT (currently defaulting to josephg's Diamond-Types). You can even read and write directly using the Braid-Chrome extension, which adds a devtools panel to view the version history of any Braid-HTTP resource: https://github.com/braid-org/braid-chrome.

I'm really excited about this library. We were going to announce it tomorrow (at https://braid.org/meeting-86), but since this hacker news discussion is happening... I can't hold myself back from talking about it now!

toomim··on Braid: Synchronization for HTTP
WebDAV was really complex, and required (ugh) locking for concurrent edits, which really sucks.

Let's compare WebDAV with Braid for something simple like looking up the current version of the resource.

In WebDAV, you have to:

- Send a PROPFIND request to the URL of the resource you want to check. In the request body, specify the DAV:version-controlled-binding property you want to retrieve, using a custom XML syntax. Here's an example:

    PROPFIND /path/to/resource HTTP/1.1
    Host: example.com
    Depth: 0
    Content-Type: application/xml
    
    <?xml version="1.0" encoding="utf-8"?>
    <propfind xmlns="DAV:">
      <prop>
        <version-controlled-binding />
      </prop>
    </propfind>
- The server responds with a 207 Multi-Status response, with XML to parse:

    HTTP/1.1 207 Multi-Status
    Content-Type: application/xml
    
    <?xml version="1.0" encoding="utf-8"?>
    <multistatus xmlns="DAV:">
      <response>
        <href>/path/to/resource</href>
        <propstat>
          <prop>
            <version-controlled-binding>
              <href>/path/to/version/history</href>
            </version-controlled-binding>
          </prop>
          <status>HTTP/1.1 200 OK</status>
        </propstat>
      </response>
    </multistatus>
This does not yet give you the version -- it tells you a separate URL that contains the version history. So now you have to query that for the version:

- Send a REPORT request to that url, and in the request body, specify the DAV:version-tree report you want to retrieve:

    REPORT /path/to/version/history HTTP/1.1
    Host: example.com
    Content-Type: application/xml
    
    <?xml version="1.0" encoding="utf-8"?>
    <version-tree xmlns="DAV:">
      <prop>
        <version-name />
        <creator-displayname />
        <creation-date />
        <comment />
      </prop>
    </version-tree>
- The server responds with a 207 Multi-Status, with more custom XML for you to parse:

    HTTP/1.1 207 Multi-Status
    Content-Type: application/xml
    
    <?xml version="1.0" encoding="utf-8"?>
    <multistatus xmlns="DAV:">
      <response>
        <href>/path/to/version/1</href>
        <propstat>
          <prop>
            <version-name>1.0</version-name>
            <creator-displayname>John Doe</creator-displayname>
            <creation-date>2023-05-26T10:00:00Z</creation-date>
            <comment>Initial version</comment>
          </prop>
          <status>HTTP/1.1 200 OK</status>
        </propstat>
      </response>
      <response>
        <href>/path/to/version/2</href>
        <propstat>
          <prop>
            <version-name>2.0</version-name>
            <creator-displayname>Jane Smith</creator-displayname>
            <creation-date>2023-05-27T14:30:00Z</creation-date>
            <comment>Updated version</comment>
          </prop>
          <status>HTTP/1.1 200 OK</status>
        </propstat>
      </response>
    </multistatus>
You can find the versions in there, and if you then parse and filter to the most recent version, you can see that the current version is 2.0.

That was a LOT of work!

Now let's see it in Braid.

To get the current version, just do a normal GET or a HEAD, and look at the Version: header in the response:

Request:

    GET /path/to/resource HTTP/1.1
Response:

    HTTP 200 Ok    
    Version: "2.0"

    <body of resource>
That's it! It's that simple!

Braid is this simple because it is baked into HTTP. It turns out that HTTP itself is already very close to a State Synchronization protocol -- it just needs a few headers, and the ability to stream updated responses when a resource changes. I encourage you to check out the spec, and see how easy it is to do things that were complicated or impossible in WebDAV:

https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-b...

toomim··on Braid: Synchronization for HTTP
HTTP (HyperText Transfer Protocol) is a State Transfer protocol. ReST (Representational State Transfer) is a State Transfer architecture. It was built for transferring a page from server to client. But if the page changed afterward, the protocol threw up its hands, and left it to the user to click "reload."

Braid proposes new HTTP headers that give HTTP new features, so that instead of just being a State Transfer protocol, it gains the functionality to become a full State Synchronization protocol.

To "extend" a protocol means to add new features to it, e.g. with new headers, which doesn't break existing uses of the protocol, but allows new implementations to opt into new extended functionality.

HTTP was originally stateless, but then added cookies. Over time, web apps evolved to store tons of state in databases on the server. Then they started storing tons of state in javascript variables and DOM on the client. Today, most of the code we write in a web app is there to synchronize the changing state on a server with the changing DOM on the client. The web is no longer static pages. It is dynamic state. But since we haven't extended HTTP to support dynamic state, programmers have to write all this synchronization code by hand. This has been a pain in the ass, and so we've evolved monstrous stacks of Javascript frameworks (react, redux, etc. etc.) to try to help manage this state synchronization from server to client. If we build this functionality directly into HTTP, tons of code goes away, and the web itself gains great new features. Then, since we will have a standard for state synchronization, our state can interoperate, and walled siloes of websites break down. We can have a P2P web.

When this work is complete, the browser will no longer need a reload button. The browser/server will guarantee that every page is always up-to-date, and every peer that synchronizes with that state will have an equal copy.

toomim··on Braid: Synchronization for HTTP
Not sure where you're seeing "limited to Lua and Haskell."

Most of the implementations are in Javascript. Check out braid-text, it's super practical: https://www.npmjs.com/package/braid-text

There are also implementations in Rust and Go.

toomim··on Show HN: Hyperdiv – Reactive, immediate-mode web UI framework for Python
Very cool! I resonate with those values!

It also allows standard tools and front-ends to work with any back-end. We could have CDNs that can cache and sync dynamic HTML for a set of users without touching the server.

toomim··on Vice website is shutting down
Amazing references! Thank you!
toomim··on Rainwater everywhere on the planet is unsafe to drink due to chemicals (2022)
I just take some sheets of aluminum foil, fold them into a hat, and then jump inside and hide from the world.
toomim··on Institutions try to preserve the problem to which they are the solution
You're losing the point. It's not outrage at the amount they are paid. It's outrage at the Shirky Principle-- they are incentivized to keep the homelessness problem going and growing.
toomim··on Vice website is shutting down
I've been wondering this too. I've even been considering authoring a web standard to allow hosts to specify how their pages can be archived in a standard way (e.g. which scripts to include, etc.) and then pitch the IA to offer a "pay $X to archive this data forever" deal to the universe.

I'm really curious what the cost per byte would be to make it worthwhile to offer a "host this byte forever, for one up-front fee" service.

toomim··on Show HN: Hyperdiv – Reactive, immediate-mode web UI framework for Python
Can you explain why you'd want plain HTML over the wire?

I happen to be working on a standardized protocol (braid-http) for sending diffs over HTTP, and have been thinking about applying it to hyperdiv. I'd love to understand your use-case and what about plain HTML you like here.

toomim··on Show HN: Hyperdiv – Reactive, immediate-mode web UI framework for Python
> although the audio is superfluous.

It's so funny how subjective that is — I liked the audio enough to tell the author about it in a DM. Then I come to HN and read that someone disagrees. :)

toomim··on FCC rules AI-generated voices in robocalls illegal
If your car gets stolen, and the police find it, they will call you from a phone number that's not in your contacts. If you don't pick up, you won't realize that your stolen car has been recovered a couple miles from your house, and if you show up there in 30 minutes you can drive it back home, but if you don't, the police will send it to a towing yard, which will require you to go through 24 hours of paperwork with the police to obtain a release and then pay the towing yard $1,000+ to tow and store your car.

If you live in an area of low crime, though, maybe it'll be fine not to answer phone calls from numbers that aren't in your phone.

toomim··on Report on the role of standardized test scores in undergraduate admissions [pdf]
Um, 22% > 1/5.
toomim··on 'Stupid,' 'shameful:' Tech workers on Y Combinator CEO Garry Tan's rant
Not with rent control. If we're in a low price market, it and a renter snags rent at a low price, he gets you locked in at that low price even when the market recovers.

Rent control incentivizes empty housing during bear markets.

toomim··on Show HN: Mailready – meet the new email deliverability standards
Maybe because they click a link in the confirmation email?
toomim··on Patients say keto helps with mental illness. Science is racing to understand why
And fasting has been a part of human spiritual (read: "psychological") practices for millennia. It's no surprise that it does something to the psyche.
toomim··on Faircamp is a free Bandcamp alternative
Oh. That's a big deal.
toomim··on YouTransfer: Self-hosted file transfer and sharing solution
And for non-technical folks I use wormhole.io.
← PreviousPage 6 of 34Next →