The federated code forge
tangled.org
tangled.org
One musing: I did run into my first "issue" with federated Git hosting the other day while setting up Neovim on a new machine. One of the plugins I use is hosted on Tangled and I guess its backing Knot was down. This was awkward for two reasons:
1. It as unclear to me from the Tangled website the backing knot was the issue (for a while I thought it was something on my end)
2. I really had no backup plans for something like a Neovim plugin's source forge being down and, clearly, neither did the Neovim package manager as it just kept showing an error on every session I started with no way to fix it :).
I suppose in the long run we really shouldn't be assuming that code storage is just Always There or easily cached somewhere, but I anticipate some growing pains in the mean time. ATProto doesn't seem to have the benefit ActivityPub has where if the instance I wanted to view content on was down I can usually find some other instance that's gotten the messages.
Pretty much. We want to make running your own git remotes as easy as possible. Lets you scale on your own too.
> ATProto doesn't seem to have the benefit ActivityPub has where if the instance I wanted to view content on was down I can usually find some other instance that's gotten the messages.
It does give you (or anybody) the ability to index this data independently, and not miss messages -- which AP doesn't afford. Different tradeoffs!
Someone (or some group) could also in theory go fully content-addressing with their data store, and make content available based on their hash, effectively layering that missing benefit on top of the existing architecture. Someone could probably vibecode this together in some hour(s) :)
I do agree that the sweet spot in federated git hosting could be a system merging the features of tangled with the features of radicle
But I have to admit that the design tradeoffs are really different
this problem could exist with any package manager, the go package manager is one example, where it allows specifying packages from any git url. we host a mirror of all git repositories on our infra, and serve that when the backing knot is down. you could choose to trust that and pull from there (verifying that the tarball has a known correct hash), the way the go package manager pulls from proxy.golang.org.
But yeah I recognize it's an issue for any package manager. I suppose I just forgot that even something like my local editor is reliant on remote forges now unless I go out of my way to mirror it all.
Happy to answer any questions at all.
So what’s the state of Tangled today? Say I have a Tangled server (instance? host?) up and running with my repos. Some stranger wants to contribute. What do they have to do before they can open issues or send PRs or whatever?
Your Tangled git server that you'd want to run is a 'knot'[1]. It's not exactly analogous to Forgejo because it's headless, but that makes it super light on resources to run.
[0]: https://atproto.com/guides/the-at-stack#pds [1]: https://tangled.org/docs/knots/self-hosting/
This is great. I'm delighted at the idea of having my own knot hosting my projects, talking to my friend's knot, and coordinating on a global project's knot, all without having to lock everyone into one centralized point of failure (even mine). Amazing. I love it!
You have options for self-hosting one or all of three pieces: a headless git host (the knot) which you get direct ssh access to through a limited ssh interface like other forges use, a CI service (the spindle), and the service that provides your user identity and metadata (the PDS). Account metadata such as which repos are in your account, and public SSH keys are recorded in the PDS, while repo data such as pull requests are recorded in the knot (I think. There might be some cross-over for publishing user activity via the PDS, I'm not sure).
Tangled owns the frontend, which everyone accesses through tangled.org. That's the web interface for both git features, and for CI. The frontend is an "AppView" which is difficult to self-host because it requires ingesting and aggregating a lot of network data. That's what assembles the network-wide activity feed you see on tangled.org, and that's where "Bobbin" comes in. But you _could_ access everything through a different AppView because your identity, data, and metadata are all owned by your PDS and knot.
The AppView concept is a point where Atmosphere differs from the Fediverse. In all my Fediverse experiences the service that I log into is the same service that runs the app frontend. With Atmosphere an AppView uses OAuth to authenticate you with your PDS. So I can log in to both Bluesky and Tangled with the same identity, compared to Fediverse where I need different Mastodon and Lemmy accounts to use both "apps". (Bluesky is also an AppView btw.) Instead of many federated frontend+backend instances, Atmosphere tends toward many federated, headless backends, with a small number of centralized frontends.
The PDS provides your DID which is your unique identifier in the atmosphere. It seems like the knot, in addition to hosting git repos and providing SSH access, does some PDS-like things. The knot provides DIDs for your repos, and holds metadata. But again, the metadata in the knot applies to the repo - metadata that applies to you as a person lives in your PDS.
I'm self-hosting a knot, and I'm planning to self-host a spindle. But for the moment I'm using a PDS provided by Bluesky.
soon, repos will be able to declare a contribution policy: presently its open to all (anybody can open an issue), but owners can choose to limit this to just contributors for example.
Also, will Tangled support private repos that are only visible or fetchable to specific people? Would that even make sense in its model?
private repos are coming very soon, we have a working demo of private repositories.
i just found this too, more recent, possibly better! https://tangled.org/darko.mx/tangledart
From my very limited testing private repos were the big missing piece; interested in other areas yet to be filled in.
What can I expect as a developer using Tangled to build (and deploy) my app? What is the native platform? What are typical other CI systems that have been adapted to work on Tangled (Dagger, Drone, whatnot, especially if there are repos to demonstrate the adaption)? What can I expect as enterprise or lone developer using Tangled or self-hosting or federation?
The system is called Spindle, but what about it? What does it take inspiration from, what are things like it that would help us spool up on this?
The way that spindles work is that they basically just get events from repos when something happens, like a push or whatever. The spindle, if it's going to respond, eventually responds with what the results were, if any. If it looks like a spindle and quacks like a spindle, it's a spindle.
Tangled's CI happens to do this by way of receiving an event, looking at your Tangled workflow files, and using those to determine when it should actually run your CI. There's no reason it has to stay like that though.
I personally am planning on implementing a Tangled spindle that, instead of looking at Tangled workflow files, looks at just my Nix flake and builds and deploys things based on that. It's a very open solution, everything is just a protocol, there's no lock in.
Tangled's Spindle itself, as a product that conforms to the informal Spindle protocol, is pretty decent. You put in Nix dependencies and some scripts and when it should run, and you get out working CI. I have no complaints about it
its undocumented at the moment but you push over https as well.
somebody has set up a git credential helper here: https://tangled.org/pds.dad/tangled-core/blob/feature%2Fgit-...
Back when you first launched, I thought running a knot was already pretty easy :) Could you expand on how it's different now, and what part(s) aren't now easy to self-host, if there is any?
https://tangled.org/docs/self-hosting/indexer/
we have also stabilized lexicons, so if you are a selfhoster, you can expect to not require a breaking upgrade!
git clone https://tangled.org/tangled.org/core
Cloning into 'core'...
warning: redirecting to https://knot1.tangled.sh/did:plc:j5hmlfdrwkvtxm7cjmu7j2is/
remote: Thanks for using knot1.tangled.sh!
remote: Enumerating objects: 99191, done.
remote: Total 99191, done.
Receiving objects: 100% (99191/99191), 122.19 MiB | 39.12 MiB/s, done.
Resolving deltas: 100% (49286/49286), done.
error: invalid path 'localinfra/fixtures/blobs/did:plc:igzuueb63dytyy3tpaweofuu bafkreieify5bix4cmrj4iuxiceb6srr2boltkbbf7yizwtgwurqv42sltm'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
and retry with 'git restore --source=HEAD :/'
You can inspect what was checked out with 'git status'
1. Redirects from .org to .sh suspiciously
2. falls over.
Try cloning on WSL, in the VM's filesystem, and let us all know how that goes.
I just wanted to read the source locally. I know things are getting worse by the month for developers on Windows because other developers are increasingly not giving a shit to support it, but it's odd to have a repo that doesn't even clone on windows.
Not building is common, not passing tests even if it builds is extremely common, but not even cloning is a new one.
I guess eventually I'll give up and switch to linux, but I'm still paid to work on Windows. I guess maybe it's finally time to look into a real dev-container setup for this kind of thing and I can do this all a bit more securely too.
For my visual impairments, macOS is currently fine, because I only really need magnification and I can get away without using much of it since everything I really deal with is free flowing text, like in a terminal or IDE. But as I need to rely more on screen readers, I anticipate I may have to consider switching. Switching to something other than Linux would break my heart, though.
Some feedback:-
* I found the UI on the homepage/timeline a bit busy - lots going on. Notifications, Recents, Announcements don't seem like they need to live on this page?
* The Activity chart on a users profile page seems a bit wild to me. I'd suggest simplifying it or just using what GitHub/GitLab use with their square grid activity chart - it's what users are most familiar with. It's kind of analagous to all the LLM/Chat applications in recent years - they all converged on the same UI because that's what works - no need to reinvent the wheel when users understand an existing pattern. Plus - it's better for marketing:- people like to share their activity on GitHub/GitLab as it quickly conveys how much work they've done over the year - your current page doesn't.
I am asking because I have been waiting for that change. I intend to make all the public repos from https://juju.bi (code forge I am building) interoperable with Tangled and basically behave as knot server.
Pre-update, code review UI was horrific compared to GitHub. Post-update, code review is much better, but "Comments" section without a "Comment" button is unforgivable. People are probably closing PRs right and left without knowing it. Shift-Enter is a submit action.
I'm a keyboard-first user, but for something which can end up in somebody's inbox notifications, UI button is the way to go.
I just don't understand how such a prolific pattern is being ignored.
I shared my vision for what it could have been in the first ATProto Private Data WG: https://www.youtube.com/watch?v=oYKA85oZc8U&t=3812s (basically Zanzibar, UCANN is also interesting) I have ideas for a new protocol that would be permissioned, more federated, but still draw on the good parts of atproto. It's geared towards small social (think discord server / facebook groups) than big social (most common apps we think of - with reposts, quotes, blocks - features I have sworn off).
The Bluesky team didn't join the conversation until pretty late and then picked their own proposal (blog post series, was not formal at that point).
Huh that's surprising to hear? What makes you think that? It feels like the whole point of this release is to make it easier to allow people to self host or federate whatever parts they want? That's what I got from their intro/opener!:
> But the centralised forge is crumbling. Relying on someone else for uptime, for features, and to be a good steward of your code hasn’t played out well. What has stood the test of time is email: Git to move the code, and email — an open protocol — to talk about it. At the cost of user experience, however.
> But the centralised forge is crumbling. Relying on someone else for uptime, for features, and to be a good steward of your code hasn’t played out well. What has stood the test of time is email: Git to move the code, and email — an open protocol — to talk about it. At the cost of user experience, however.
> Tangled aims to give you the best of both: the longevity of an open protocol, with the convenience and polish of a proper forge.
> It’s the first truly federated code forge — you can host code on your own servers and still collaborate seamlessly with others. Our approach is pragmatic: self-host just the parts you want, like Git hosting or CI runners, or all of it, including the API server and UI. Whatever you choose, you can still talk to everyone else.
The comment here from @icy emphasizes similar ability to let whomever host whatever parts of this they please:
> Hi! Tangled co-founder here. We're super stoked with this launch -- we've stabilized our AT Protocol lexicons, completely overhauled our architecture and UI, and made self-hosting the entire stack (or parts of it!) https://news.ycombinator.com/item?id=50306205
I don't see any mention of "their struggles around scaling centralized infrastructure". Is that just typical run of the mill low grade slander attack we see against atproto, more reflexive negativity? Or is there some actual interesting point here, that is perhaps even relevant to this tangled release?
Code hosting platforms are probably much more disjointed than microblogging platforms, so something like ActivityPub, where servers only keep track of content that users are actually interested in, would make much more sense. It's a shame that the people working on ForgeFed and/or Forgejo federation seemingly aren't making much progress.
First, Tangled has been entirely self-hostable since the beginning! We've just made it much easier to run for everyone.
Second, the "appview" is simply an indexer. It consumes records from the firehose and only those that it cares about. In our case, those are org.tangled.*. There's no reason we'd consume all of the firehose.
ActivityPub categorically does not make sense here because of how coupled identities are to instances.
As has BlueSky, as long as you had enough hardware to keep up with the firehose. The protocol heavily lends itself to centralization due to this design. I think this should be obvious by the fact that you haven't considered self-hosting to be the default case.
> In our case, those are org.tangled.*.
I suppose my wording was inaccurate, but this is what I meant by "the entire network". My indexer would have to consume "all of Tangled" instead of the things related to the repos, orgs and people I am interested in. Correct?
> ActivityPub categorically does not make sense here because of how coupled identities are to instances.
I feel like this is more of an inconvenience rather than a dealbreaker. Having to point people to a new account ID on the off-chance you lose your domain seems like much less of a problem than having to deal with ATProto's firehose architecture.
We've already explained to you twice that this is incorrect, you don't need to do this. And that if you do want to do this for some ridiculous reason, just because, a $5 VPS can do it.
But you do acknowledge latter you are only talking about tangled records.
Currently very few people are worried about this problem, because all of the tangled traffic just isn't that big, and you can drop most of it very quickly with little impact. Relays do allow various kinds of filters though. You could use a service like Microcosm Constellation https://constellation.microcosm.blue/ that lets you find only records for your repos etc. That would work fine for finding the records pertinent to you!
No one does this right now, because the premise is faulty. There just really isn't any difficulty or cost of any note to having all the tangled records piped at you. Hopefully some day this gets worse, and that this becomes a good problem to have! One we already have tools (like constellation) to tackle.
In short: constellation is a (also easy to self host cheap light weight) service for reverse indexing, for finding records pertinent to you. Lots and lots of atproto apps use this for this kind of "find people making records about my stuff".
A ridiculous reason, such as being able to interact with the federated world from my own server directly, and not through a third-party service. Why is the simple concept of going to your own server to interact with the entire network, getting content directly from your friends' servers, and receiving interactions (e.g. PRs) directly from someone else's server such an afterthought in ATProto?
You can host your own data very cheaply. If you don't have to serve every single other server, and keep them all up to date, and can just pump some records out at aggregating relays, it makes it much easier for people to be online. You can run a massive PDS on a tiny ancient rpi.
Which is what activitypub doesn't do. It's protocols are very heavy and hard and stateful. Running mastadon or other servers requires peering with every other related server. And it's on you to deliver messages, to push, to track state and progress. All the network work is on you to get your message out. So your server with your data has a ton of work to do. This doesn't scale well. See again Paul's scaling with open social.
If you do want to run your own atproto services, yes, you need to pull from the network to see what is relevant, you need a relay server, you need a pull. And you can! Running those relays is very cheap! The protocols scale really well & is super well optimized to be efficient to do! You can just do it!
The one particular thing you keep trying to demand is what makes AP so hard, so difficult, and scale so very badly. It's the worst part of activitypub, it makes it super hard and super expensive. Switching from AP push model to the relay doing pull model is the secret sauce that makes everything super lightweight and scalable, and that makes sync so much easier.
It is a social network so I'm not sure what the issue about yes needing to read the network is. There are really good relays designed to help aggregate and filter the network. They can run on $5 VPS, processing the entire network, which is awesomely cheap, not just Tangled, which is awesome, what I would call a huge win. Paul has a talk on designing for scalability, about how it's possible to have such small boxes doing the network aggregation with so little resources, that appviews can then consume their interests from. solving for scale in open social networks, https://youtu.be/BoJnj2yPf14
The servers are definitely not keeping the whole network. The post talks to this some, but the new Knot2 retains a local copy of repo-relevant material (comments prs etc) in its own git tree, that collects what the network has done together, and makes it easy to ship/replicate, while also using that as a sync source to replicate back into atproto too, to make network visible.
Agreed that I really wish ForgeFed were showing much more clear progress. It seems like such a good idea, but it's been a number of years now and it's super murky what the progress is and where we are. Having >1 viable player in the decentralization of forges would be great. I haven't checked in a bit but I do read Forgejo release notes and my impression is it's been a very slow trickle.
There are other projects other than Forgejo implementing ForgeFed. Forgejo itself has some federation support but AFAIK not based on ForgeFed. But they have a community contrib project that does.
- https://codeberg.org/forgejo-contrib/federation
- https://codeberg.org/activitypub-rs/activityforge
- https://frogue.vit.am/~ololduck/Frogue
Gitlab has a community epic to add ActivityPub support, but that needs contributors:
Generally you don't have to do this! There's lots of people running relays that are open access. But if you want your own it's very cheap. List of relays: https://tangled.org/firehose.club/community-firehose-list
Can you also fit a PDS to host yourself on on that VPS? Yeah I expect that it would fit very easily. PDS's have a lot less traffic hitting them than ActivityPub, they can be way smaller. They are quite lightly loaded! Because... there are relays in front of them!
atcr.io/tangled.org/knot:2
> Radicle’s Collaborative Objects (COBs) provide Radicle’s social primitive. This enables features such as issues, discussions and code review to be implemented as Git objects. Developers can extend Radicle’s capabilities to build any kind of collaboration flow they see fit.
Radicle has some association with cryptocurrency, which is a bit ick to me, but as far as I understand it, it's actually a non-profit and the crypto was more of a grant than buy-in.
I do, however question the taste and judgment of any project that tells users to run `curl ...|sh`.
At this point it's really the only forge I want to use going forward, considering that it sits nicely alongside all my other social data with ATProto.
AI would make it easier to implement and test complex protocols like this, but the community using Forgejo is mostly against it, which means it'll likely stay in limbo for a long time.