HNHacker News
TopNewBestAskShowJobs

pmeunier

672 karma · joined July 13, 2014

submissionscomments
pmeunier··on More Productive Git
The main point of Pijul is, you never have to think about categories when using it. It's just a sort of Git where you almost never need to branch (the equivalent of a Git branch in Pijul is just a patch), merges are always deterministic and easy to understand, cherry-picking is the default, and works even for conflict resolutions.
pmeunier··on Oh shit, git (2016)
It's indeed not possible to make merge mistakes in Pijul, in the sense that the order between lines is always preserved: if line A comes before B in one repository, A will be before B in all repositories that have these two lines.

Moreover, merge is associative, which is the intuitive idea of a "good merge", in the sense that merging your patches one by one is the same as merging them all at once (this is false in Git).

In Pijul you can't "make mistakes rebasing". A repository is a set of patches (set as in maths), you can add patches or remove patches from the set, and that's about it. Two repositories with the same set of patches (possibly applied in different orders) are totally equivalent (in particular they have the same file contents).

So, rebasing in this case would be just the operation of adding some patches and removing others, and hence you can't make mistakes doing that, because there is no manual merge operation needed (of course you still need to solve your conflicts).

pmeunier··on Oh shit, git (2016)
I'm one of the authors. Pijul is indeed heavily inspired by Darcs, but we solved that problem completely.

All our algorithms are in time at most log of the size of history, which is a double-exponential improvement over Darcs' worst case complexity.

pmeunier··on Oh shit, git (2016)
> The FAQ even says, “Pijul is trivial for whoever knows category theory.” Is that question really asked frequently? Words like that might convince me to never try it. ;)

There have been a few blog posts after we first announced it saying "Oh, it's just category theory, so I could rediscover it independently in Haskell in just a few hours".

pmeunier··on Oh shit, git (2016)
I'm one of the authors. Our goal is to make something much much easier to use than Git, and we're already succeeding in doing that.

The math can be completely ignored by the user, it is just a way to guarantee that the tool will always match the basic intuition of version control. For instance:

- "Associativity" is a concept of algebra, but when applied to version control, it just means the following: let's Alice makes a commit A, and Bob makes two commits B and C. If Alice pulls B, and then pulls C, associativity means that she'll get the same as pulling B and C together. Pijul is associative, but Git is not!

- "Commutativity" is also a concept of algebra, but in practice in version control, it means that you don't have to think about feature branches. In Pijul, two patches that you could produce on different branches are "independent" anyway. Applying them in any order will always yield the same result, so you can push any of them, no matter the order in which you made them. This property means that "rebasing" becomes much simpler: it's just the action of applying and unapplying patches. Also, cherry-picking becomes the default, you don't even have to think about it.

- "Inverses": in Pijul, all patches have inverses. Sure, it sounds like `git revert`, except that `git revert` can sometimes screw things up very badly (if you `git revert` a merge commit, for instance).

Also, in Pijul, conflicts are the normal state, so you don't need anything like `git rerere`, and two users with the same patches (even in different orders) will always see the same conflicts. If you have a patch solving a conflict between two other patches, then the conflict is solved forever, and doesn't "come back".

As a conclusion, I'd say Git might be too technical for a normal programmer (which is why we have giant threads like this one on HN), but Pijul is the exact opposite.

It is as powerful as Git, but beginner-friendly, infinitely faster to learn, and more flexible. Since there can be no "bad merge", you don't have to think about your version control system anymore, and can focus on your work.

That said, there are still a few problems:

- it's not yet as efficient as Git for storage, but we're working on it. We just released the first full implementation a few weeks ago.

- it will never be as good as Git for detecting file moves, because Git is happy with a soup of blobs, whereas Pijul needs more structure (but OTOH that extra structure makes Pijul much better at "blame").

pmeunier··on Pijul for Git users
You seem to be faster at writing than at reading, I already answered your point 1:

> it becomes fairly clear if you try to cherry-pick more than twice from the same branch, and find conflicts on the way. Or when you want to "undo" a merge, and push that undoing to a remote repository.

Any team with enough people, or working at a fast enough pace, knows this doesn't work in Git. Pijul completely solves it by thinking in an entirely different way (changes instead of snapshots). This is not an implementation detail at all, the two mental models are extremely different.

About your point 2, what can I say? You picked the example, and now that you're not liking it anymore, you are trying to blame it on me? I don't find that fair.

pmeunier··on Pijul for Git users
> I would say that's less fundamental than answering "give me the exact state of the repo at time t"

This is indeed a very important question when you're working alone, but the notion of "at time t" changes when many people work in parallel. This is why Pijul has two notions of time:

- Patches, which always commute when produced in parallel. - The sequences of patches applied locally to a branch.

Alice and Bob working together can have the same patches and a different application order, what matters for the contents of their file is the set of patches only.

> Of course the hashes won't be the same but does that matter?

It does matter a lot sometimes, for instance if you cherry-pick, solve a conflict, and cherry pick again from the same branch, the fact that Git changes the hashes will make the conflict reappear the second time you cherry-pick.

pmeunier··on Pijul for Git users
No, it's actually related to the ease of use and correctness. By design, Git lacks two fundamental properties that would make working with repositories much easier:

1. Git merge is not associative. This results in cases where Alice and Bob work together, Bob pulls Alice's work, and her lines get merged into places she's never seen (blocks of text newly introduced by Bob).

2. More importantly, Git is not commutative. This means that cherry-picking and rebasing are complex operations that change commits' ids. This forces Git users to branch (an extra step that needs to be done before starting to write), but more painfully, it sometimes forces them to do multiple steps of rebasing before being able to merge what they want, or to solve the same conflict again and again after an unlucky cherry-picking.

Of course, all this could maybe be prevented by stricter planning and a more vertical organisation. But this is not how many people write code in 2019. Continuous delivery, for instance, means that teams no longer know long in advance what they are going to over the course of the project. Also, the best developers don't work in vertical teams where they get continuously told what to work on.

pmeunier··on Pijul for Git users
The theory is actually quite different, even though Pijul and Darcs behave in a similar way on many examples.
pmeunier··on Pijul for Git users
> the mental model will require some mental rewiring

One must be pretty brave to admit that. Even though I'm a big user of Git (and a co-author of Pijul), I agree that you'll probably need to unlearn a number of things. But you won't need to relearn so much, the patch model is really more intuitive in many cases.

pmeunier··on Pijul for Git users
One of the largest open source repositories ever:

https://github.com/NixOS/nixpkgs

pmeunier··on Pijul for Git users
> That commit id is a snapshot, even if it is stored as a patch.

I agree with your argument, but Git does store "blobs" only (plus "trees"), and never patches. When you can ask it to show you a diff between two snapshots, Git computes the diff on the fly.

More almost-accurate info here: https://git-man-page-generator.lokaltog.net/

pmeunier··on Pijul for Git users
By the way, that guide is not from the authors of Pijul, even though as an author myself, I'm happy to see people understand it, use it and explain it.

I believe a more gentle and constructive way of discussing this is by starting a discussion on their page [1].

[1] https://nest.pijul.com/tae/pijul-for-git-users/discussions

pmeunier··on Pijul for Git users
> That git works as a blob store internally (through tree objects) is, to the user, an irrelevant and hidden implementation detail.

1. Sorry, I get that you might be angry at Pijul for some reason, but this is simply not true. The lack of associativity of 3-way merge is absolutely not hidden, and gets merges wrong sometimes.

But more fundamentally, the fact that commits must be ordered is not an implementation detail, it becomes fairly clear if you try to cherry-pick more than twice from the same branch, and find conflicts on the way. Or when you want to "undo" a merge, and push that undoing to a remote repository.

Conflicts in Pijul are detected correctly, and a conflicting state is another valid state, whereas in Git, a conflict is a failure to merge, and the user must solve the conflict before doing anything else. In particular, this implies that a conflict resolution in Git can be propagated to other repositories only if the two conflicting commits were merged at the exact same time in all repositories.

2. Working on parts of the repository in Git only works for checkout, you still have to clone the entire repository, and Git still has the entire repo, even it is not shown to the user in its entirety. This completely defeats the point of partial clones, and prevents, in many cases, the use of Git on monorepos. That said, I'm not sure why this is the case, Git could very well produce a commit using a tree whose blobs are not all downloaded, just like when you commit from a shallow clone.

But even if this worked, Git requires you to send the entire history of the full repository if you want to clone it to a remote machine, whereas Pijul lets you select the patches relative to just a part of the repository. This is because in Pijul, patches commute! If you place your patches in a sequence, you can reorder the patches you want to send (without changing their identifier) to be the first ones in the sequence, and then send just a prefix of the sequence. All this without any extra operation, rebase or anything else.

pmeunier··on Pijul for Git users
1. It is associative, which make it predictable, and yes, better.

2. It is commutative, which makes it non-trivial to emulate with Git, a system where nothing commutes (on purpose).

Of course, you can obviously do your merges manually, rebase the world at each new commit, cherry-pick and rerere to emulate what Pijul gives you for free. But then is your job writing code or showing your proficiency at Git?

pmeunier··on Pijul for Git users
May I suggest maybe reading the manual before writing this kind of comment?

https://pijul.org/manual/why_pijul.html

pmeunier··on Pijul for Git users
By having a mathematical theory of patches instead of a bunch of (user-friendly and often working) hacks. It's actually explained in the manual:

https://pijul.org/manual/why_pijul.html

pmeunier··on Pijul for Git users
I'm one of the authors. We are actually quite optimistic about Pijul's possibilities. In particular, there are a few niches where people either:

(1) do not understand Git, can't learn it, still need a VCS. I have a few industrial applications in mind, not necessarily related to code.

(2) understand Git very well, and have spent too many hours debugging their failed merges, non-associativity issues, complex rebases. These users would know enough about "git rerere", for instance, to find the model of conflicts slightly absurd. For them, Git has become a source of cost in the project, and costs need to be kept under control. Again, I have at least one open source project in mind, and a handful of industrial contacts related to these use cases.

(3) many people in academia, even in computer science, collaborate on their LaTeX papers using various combinations of Dropbox and emails. This is because they are "principles people" and like to understand the principles behind what they're using. If a tool seems too complicated, it looks suspicious. But anyway, I have written papers with Git before, the "fun" of using Git often fades away as the deadline approaches.

Given the very young age of Pijul, I see this picture, and the interest many people have shown in the project, as quite promising. However, unlike what your post may suggest, I don't see Pijul as fighting against Git. Ultimately, when Pijul is ready, the two tools could become quite complementary:

- use Git (with branches, but without merges) when you need a linear history and/or a Merkel tree of your content. Git is extremely efficient in disk usage and in speed of retrieving contents. Projects like "Software Heritage" are a good example of the perfect Git-but-not-Pijul user, storage of experimental data is another one ("I've done experiment X on day Y, the results were Z").

- use Pijul when you want to collaborate efficiently on text documents, and manipulate your repository in an extremely flexible way, handle conflicts sanely, while minimising your planning and maintenance costs. Pijul will probably never be as efficient as Git in terms of disk space, but could become quite competitive in terms of merge speed. Agile teams, academic papers, are good examples.

pmeunier··on Pijul for Git users
Sure, but if you had read the manual[1], you would have learned that this is only the associativity part, which you could indeed use to provide a better merge algorithm in a snapshot-based VCS.

So:

1. You're 100% correct in saying that this can be done in a snapshot-based VCS like Git, Mercurial or SVN.

2. But you're also missing the main feature of Pijul, patch commutation, also explained in the manual. The idea is that you don't need to rebase anymore, two patches produced independently always commute, and hence patches don't need to change their identity when they are "rebased", or merged at any point in time. This has major usability benefits, such as being able to undo a "merge" (I prefer to call it a patch application) even after ten other "merges". Or choosing which part of a conflict to remove in order to solve it (if you don't choose to actually solve the conflict). With patch commutation, you don't need `git rerere` anymore, and you don't even need to branch very often, since branch are essentially an emulation of commutation (then of course Pijul still has branches, but they're quite different from Git's branches, in particular they are not as essential as in Git).

But then Pijul is still a young project, there are still a few bugs here and there. It is self-hosted, and we're quite happy using it for its own development (and the development of all companion libraries, Thrussh, Sanakirja…). But is it really ready for HackerNews-level flamewars?

[1] https://pijul.org/manual/why_pijul.html|the manual

pmeunier··on Pijul for Git users
I'm one of the authors, and we've done a lot of bibliography before engaging in such a big project. I'll talk only about the core principles behind version control, not about how things are actually implemented, or which options are the defaults in which tool.

Pijul is absolutely not how RCS or CVS worked. Instead, RCS and CVS worked more or less like SVN (remember the "CVS done right"?), and SVN itself is not too different from "Git without branches", or "Centralised Git", in the sense that these tools are snapshot-based (even though the actual implementations may store patches sometimes). This means that they see history as sequential, and use algorithms to merge when necessary. One could actually very well simulate a large part of Git branches and merges using multiple SVN servers, even though that would obviously be quite cumbersome to implement and use in any concrete situation.

The only version control system one can clearly relate Pijul to is Darcs, using the idea that patch commutation is the main thing people care about. Unfortunately, Darcs has important complexity problems when dealing with conflicts, and Pijul solves these problems.

That said, short-lived branches ("feature branches") in Git mostly try to simulate commutation, i.e. independent development mergeable later, hopefully in any order (since they were written independently).

pmeunier··on Pijul for Git users
I'd say maybe check it out first, especially the parts about patch commutation and the problems with associativity in git merges and rebases.
pmeunier··on Writing a Microservice in Rust
Sounds great! One thing I've always had trouble with is detecting unused dependencies. If a project grows fast, it's easy to leave some unused modules in Cargo.toml. At least, matching them with their `extern crate` counterpart helps detect unused ones, by relying on rustc for the check.
pmeunier··on Pijul 0.3 release: a VCS using patch theory
It's that, plus a number of other things:

- Rust is low-level enough to write sanakirja, which is pijul's database backend, in a clean way, without having to decode/re-encode tons of things from disk all the time.

- Types! A sane (as in "mathematically sound") type system. That sentence is easy to misinterpret if you've not written actual programs (of more than ten lines) in functionnal programming, but types are essential for a program several dozens of thousands of lines long, in total. It basically means that we could refactor our code many times without fear.

- Rust's community is awesome. I had never seen anything like that before.

- Support for threads, both low-level and easy.

That said, Turing-complete programming languages won't prevent anyone from doing stupid things, by definition. I'm not sure about the particular focus on parsing: non-parsing tasks also required a bit of thinking.

pmeunier··on Sanakirja: Transactional, branchable key-value store in Rust
1. Sanakirja is fully transactional.

2. Sanakirja supports fork.

3. TiKV is distributed.

4. Sanakirja was started before (the Pijul project needed something like that, and TiKV was not around).

pmeunier··on Sanakirja: Transactional, branchable key-value store in Rust
Author here! I did start Sanakirja in Finland, but I'm not a Finn, so I thought I was allowed to break that rule.

Also, getting traction for Sanakirja has never really been a goal, at least not as much as getting something that we could use in Pijul.

pmeunier··on Pijul 0.3 (VCS): first public release
Fixed on nest.pijul.com/pijul_org/pijul, thanks for the report.
pmeunier··on Sanakirja: Transactional, branchable key-value store in Rust
Author here. So I thought when I started the project. But when you think more about it, Rust cannot allocate in an mmapped file, so you're basically on your own, and have to do plain old C-like memory management in the file, except that unlike in C, there is the extra constraint that everything is transactional, pulling the machine's plug needs to "unfree" all freed pages, and "unalloc" all allocated pages.
pmeunier··on Sanakirja: Transactional, branchable key-value store in Rust
Author here! Thanks! Documentation is quite high on our priority list, actually just after fixing a few security issues on nest.pijul.com.
pmeunier··on Sanakirja: Transactional, branchable key-value store in Rust
Good question. The structure is a B tree. When you traverse it, and arrive to a block, but don't see it in the reference counting base, you know that it is referenced in the tree, so its reference count is at least 1.

Then, if you want to free it, you have to delete all references to it in the tree. Maybe that blog post could have been more verbose about another feature of Sanakirja, the list of free pages.

pmeunier··on Pijul – A free and open source distributed version control system
If you try to analyse merge systems mathematically, git's merge system is the "magical" one. It is just a heuristic algorithm, with no solid property you can rely on.

In contrast, Darcs and Pijul's merge are associative, and Pijul's merge is commutative. Even if you don't like maths, this means that they will always behave deterministically. This also means you can use them in scripts, although darcs might sometimes have performance problems (pretty bad ones, actually).

In git, you can get the following: https://tahoe-lafs.org/~zooko/badmerge/simple.html

← PreviousPage 9 of 10Next →