672 karma · joined July 13, 2014
Moreover, merge is associative, which is the intuitive idea of a "good merge", in the sense that merging your patches one by one is the same as merging them all at once (this is false in Git).
In Pijul you can't "make mistakes rebasing". A repository is a set of patches (set as in maths), you can add patches or remove patches from the set, and that's about it. Two repositories with the same set of patches (possibly applied in different orders) are totally equivalent (in particular they have the same file contents).
So, rebasing in this case would be just the operation of adding some patches and removing others, and hence you can't make mistakes doing that, because there is no manual merge operation needed (of course you still need to solve your conflicts).
All our algorithms are in time at most log of the size of history, which is a double-exponential improvement over Darcs' worst case complexity.
There have been a few blog posts after we first announced it saying "Oh, it's just category theory, so I could rediscover it independently in Haskell in just a few hours".
The math can be completely ignored by the user, it is just a way to guarantee that the tool will always match the basic intuition of version control. For instance:
- "Associativity" is a concept of algebra, but when applied to version control, it just means the following: let's Alice makes a commit A, and Bob makes two commits B and C. If Alice pulls B, and then pulls C, associativity means that she'll get the same as pulling B and C together. Pijul is associative, but Git is not!
- "Commutativity" is also a concept of algebra, but in practice in version control, it means that you don't have to think about feature branches. In Pijul, two patches that you could produce on different branches are "independent" anyway. Applying them in any order will always yield the same result, so you can push any of them, no matter the order in which you made them. This property means that "rebasing" becomes much simpler: it's just the action of applying and unapplying patches. Also, cherry-picking becomes the default, you don't even have to think about it.
- "Inverses": in Pijul, all patches have inverses. Sure, it sounds like `git revert`, except that `git revert` can sometimes screw things up very badly (if you `git revert` a merge commit, for instance).
Also, in Pijul, conflicts are the normal state, so you don't need anything like `git rerere`, and two users with the same patches (even in different orders) will always see the same conflicts. If you have a patch solving a conflict between two other patches, then the conflict is solved forever, and doesn't "come back".
As a conclusion, I'd say Git might be too technical for a normal programmer (which is why we have giant threads like this one on HN), but Pijul is the exact opposite.
It is as powerful as Git, but beginner-friendly, infinitely faster to learn, and more flexible. Since there can be no "bad merge", you don't have to think about your version control system anymore, and can focus on your work.
That said, there are still a few problems:
- it's not yet as efficient as Git for storage, but we're working on it. We just released the first full implementation a few weeks ago.
- it will never be as good as Git for detecting file moves, because Git is happy with a soup of blobs, whereas Pijul needs more structure (but OTOH that extra structure makes Pijul much better at "blame").
> it becomes fairly clear if you try to cherry-pick more than twice from the same branch, and find conflicts on the way. Or when you want to "undo" a merge, and push that undoing to a remote repository.
Any team with enough people, or working at a fast enough pace, knows this doesn't work in Git. Pijul completely solves it by thinking in an entirely different way (changes instead of snapshots). This is not an implementation detail at all, the two mental models are extremely different.
About your point 2, what can I say? You picked the example, and now that you're not liking it anymore, you are trying to blame it on me? I don't find that fair.
This is indeed a very important question when you're working alone, but the notion of "at time t" changes when many people work in parallel. This is why Pijul has two notions of time:
- Patches, which always commute when produced in parallel. - The sequences of patches applied locally to a branch.
Alice and Bob working together can have the same patches and a different application order, what matters for the contents of their file is the set of patches only.
> Of course the hashes won't be the same but does that matter?
It does matter a lot sometimes, for instance if you cherry-pick, solve a conflict, and cherry pick again from the same branch, the fact that Git changes the hashes will make the conflict reappear the second time you cherry-pick.
1. Git merge is not associative. This results in cases where Alice and Bob work together, Bob pulls Alice's work, and her lines get merged into places she's never seen (blocks of text newly introduced by Bob).
2. More importantly, Git is not commutative. This means that cherry-picking and rebasing are complex operations that change commits' ids. This forces Git users to branch (an extra step that needs to be done before starting to write), but more painfully, it sometimes forces them to do multiple steps of rebasing before being able to merge what they want, or to solve the same conflict again and again after an unlucky cherry-picking.
Of course, all this could maybe be prevented by stricter planning and a more vertical organisation. But this is not how many people write code in 2019. Continuous delivery, for instance, means that teams no longer know long in advance what they are going to over the course of the project. Also, the best developers don't work in vertical teams where they get continuously told what to work on.
One must be pretty brave to admit that. Even though I'm a big user of Git (and a co-author of Pijul), I agree that you'll probably need to unlearn a number of things. But you won't need to relearn so much, the patch model is really more intuitive in many cases.
I agree with your argument, but Git does store "blobs" only (plus "trees"), and never patches. When you can ask it to show you a diff between two snapshots, Git computes the diff on the fly.
More almost-accurate info here: https://git-man-page-generator.lokaltog.net/
I believe a more gentle and constructive way of discussing this is by starting a discussion on their page [1].
[1] https://nest.pijul.com/tae/pijul-for-git-users/discussions
1. Sorry, I get that you might be angry at Pijul for some reason, but this is simply not true. The lack of associativity of 3-way merge is absolutely not hidden, and gets merges wrong sometimes.
But more fundamentally, the fact that commits must be ordered is not an implementation detail, it becomes fairly clear if you try to cherry-pick more than twice from the same branch, and find conflicts on the way. Or when you want to "undo" a merge, and push that undoing to a remote repository.
Conflicts in Pijul are detected correctly, and a conflicting state is another valid state, whereas in Git, a conflict is a failure to merge, and the user must solve the conflict before doing anything else. In particular, this implies that a conflict resolution in Git can be propagated to other repositories only if the two conflicting commits were merged at the exact same time in all repositories.
2. Working on parts of the repository in Git only works for checkout, you still have to clone the entire repository, and Git still has the entire repo, even it is not shown to the user in its entirety. This completely defeats the point of partial clones, and prevents, in many cases, the use of Git on monorepos. That said, I'm not sure why this is the case, Git could very well produce a commit using a tree whose blobs are not all downloaded, just like when you commit from a shallow clone.
But even if this worked, Git requires you to send the entire history of the full repository if you want to clone it to a remote machine, whereas Pijul lets you select the patches relative to just a part of the repository. This is because in Pijul, patches commute! If you place your patches in a sequence, you can reorder the patches you want to send (without changing their identifier) to be the first ones in the sequence, and then send just a prefix of the sequence. All this without any extra operation, rebase or anything else.
2. It is commutative, which makes it non-trivial to emulate with Git, a system where nothing commutes (on purpose).
Of course, you can obviously do your merges manually, rebase the world at each new commit, cherry-pick and rerere to emulate what Pijul gives you for free. But then is your job writing code or showing your proficiency at Git?
(1) do not understand Git, can't learn it, still need a VCS. I have a few industrial applications in mind, not necessarily related to code.
(2) understand Git very well, and have spent too many hours debugging their failed merges, non-associativity issues, complex rebases. These users would know enough about "git rerere", for instance, to find the model of conflicts slightly absurd. For them, Git has become a source of cost in the project, and costs need to be kept under control. Again, I have at least one open source project in mind, and a handful of industrial contacts related to these use cases.
(3) many people in academia, even in computer science, collaborate on their LaTeX papers using various combinations of Dropbox and emails. This is because they are "principles people" and like to understand the principles behind what they're using. If a tool seems too complicated, it looks suspicious. But anyway, I have written papers with Git before, the "fun" of using Git often fades away as the deadline approaches.
Given the very young age of Pijul, I see this picture, and the interest many people have shown in the project, as quite promising. However, unlike what your post may suggest, I don't see Pijul as fighting against Git. Ultimately, when Pijul is ready, the two tools could become quite complementary:
- use Git (with branches, but without merges) when you need a linear history and/or a Merkel tree of your content. Git is extremely efficient in disk usage and in speed of retrieving contents. Projects like "Software Heritage" are a good example of the perfect Git-but-not-Pijul user, storage of experimental data is another one ("I've done experiment X on day Y, the results were Z").
- use Pijul when you want to collaborate efficiently on text documents, and manipulate your repository in an extremely flexible way, handle conflicts sanely, while minimising your planning and maintenance costs. Pijul will probably never be as efficient as Git in terms of disk space, but could become quite competitive in terms of merge speed. Agile teams, academic papers, are good examples.
So:
1. You're 100% correct in saying that this can be done in a snapshot-based VCS like Git, Mercurial or SVN.
2. But you're also missing the main feature of Pijul, patch commutation, also explained in the manual. The idea is that you don't need to rebase anymore, two patches produced independently always commute, and hence patches don't need to change their identity when they are "rebased", or merged at any point in time. This has major usability benefits, such as being able to undo a "merge" (I prefer to call it a patch application) even after ten other "merges". Or choosing which part of a conflict to remove in order to solve it (if you don't choose to actually solve the conflict). With patch commutation, you don't need `git rerere` anymore, and you don't even need to branch very often, since branch are essentially an emulation of commutation (then of course Pijul still has branches, but they're quite different from Git's branches, in particular they are not as essential as in Git).
But then Pijul is still a young project, there are still a few bugs here and there. It is self-hosted, and we're quite happy using it for its own development (and the development of all companion libraries, Thrussh, Sanakirja…). But is it really ready for HackerNews-level flamewars?
[1] https://pijul.org/manual/why_pijul.html|the manual
Pijul is absolutely not how RCS or CVS worked. Instead, RCS and CVS worked more or less like SVN (remember the "CVS done right"?), and SVN itself is not too different from "Git without branches", or "Centralised Git", in the sense that these tools are snapshot-based (even though the actual implementations may store patches sometimes). This means that they see history as sequential, and use algorithms to merge when necessary. One could actually very well simulate a large part of Git branches and merges using multiple SVN servers, even though that would obviously be quite cumbersome to implement and use in any concrete situation.
The only version control system one can clearly relate Pijul to is Darcs, using the idea that patch commutation is the main thing people care about. Unfortunately, Darcs has important complexity problems when dealing with conflicts, and Pijul solves these problems.
That said, short-lived branches ("feature branches") in Git mostly try to simulate commutation, i.e. independent development mergeable later, hopefully in any order (since they were written independently).
- Rust is low-level enough to write sanakirja, which is pijul's database backend, in a clean way, without having to decode/re-encode tons of things from disk all the time.
- Types! A sane (as in "mathematically sound") type system. That sentence is easy to misinterpret if you've not written actual programs (of more than ten lines) in functionnal programming, but types are essential for a program several dozens of thousands of lines long, in total. It basically means that we could refactor our code many times without fear.
- Rust's community is awesome. I had never seen anything like that before.
- Support for threads, both low-level and easy.
That said, Turing-complete programming languages won't prevent anyone from doing stupid things, by definition. I'm not sure about the particular focus on parsing: non-parsing tasks also required a bit of thinking.
2. Sanakirja supports fork.
3. TiKV is distributed.
4. Sanakirja was started before (the Pijul project needed something like that, and TiKV was not around).
Also, getting traction for Sanakirja has never really been a goal, at least not as much as getting something that we could use in Pijul.
Then, if you want to free it, you have to delete all references to it in the tree. Maybe that blog post could have been more verbose about another feature of Sanakirja, the list of free pages.
In contrast, Darcs and Pijul's merge are associative, and Pijul's merge is commutative. Even if you don't like maths, this means that they will always behave deterministically. This also means you can use them in scripts, although darcs might sometimes have performance problems (pretty bad ones, actually).
In git, you can get the following: https://tahoe-lafs.org/~zooko/badmerge/simple.html