HNHacker News
TopNewBestAskShowJobs

pmeunier

672 karma · joined July 13, 2014

submissionscomments
pmeunier··on Commits are shapshots, not diffs
I don't think Darcs 3 was even started. Darcs 2 recently solved the exponential merge, but it is now quadratic (which is still exponentially slower than Pijul instead of double-exponentially).

I've also been really happy with Darcs, expect for two things:

- Conflicts are not handled very well, as if they were not properly stored internally. `darcs revert` on a conflict doesn't always do what I expect.

- The fact that it doesn't scale (even with the quadratic algorithm) means that big users can't adopt it, which means that too few people write tools for it (including a hosting website: I'm aware of hub.darcs.net, but there are many things missing, including on security). Darcs is hard to install on platforms such as Windows. Also, few people know it, which means that it's hard to collaborate using it.

pmeunier··on Commits are shapshots, not diffs
> why does pijul decide that the correct merge is only adding the X line after the second instance of AB?

It doesn't decide, it only guarantees that the order between lines is preserved in all cases. Also, since you seem to have "thought a lot about the problem", you probably noticed that what Git does is totally wrong, since what it does depends on whether you merge both commits at once, or one at a time!

> the same person will often want different algebraic operations in different settings.

I doubt that when Alice adds line at the top of the file, while Bob edits the bottom of the file, anyone would ever want Bob's new lines merged in the middle of Alice's new lines.

Yet, that is what Git does.

> it is quite clear from this pijul example that they haven't really thought about the problem much.

Or maybe they have, and they have thought enough about it to notice that 3-way merge is doing the wrong thing, and that trying to merge snapshot doesn't even make any sense. They might have also thought enough about Git to understand that rebase is turning commits into patches, one by one, in order to "replay them on top of the current version".

While I do agree that full versions are important for storage (and all version control systems, including patch-based ones, are able to recover full versions), they are not very useful to merge work, or to solve conflicts.

pmeunier··on Commits are shapshots, not diffs
It seems we're agreeing indeed! I understood your meant that, but I also thought you meant that snapshots were an implementation detail to simulate diffs. My point is, I don't think they're good at that.
pmeunier··on Commits are shapshots, not diffs
> If I have a 100-line file and on 'main' it changes near the top, but in my 'topic' branch it changes near the bottom, then I can cherry-pick 'topic' onto 'main' and Git will resolve the diff correctly.

That is not true: sometimes Git will take the new lines from "topic" and merge them into the new lines from "main", see https://pijul.org/manual/why_pijul.html

> You might hit a conflict in your "git cherry-pick" command which gives you an opportunity to resolve the unexpected diff issue in an appropriate way, which ends up with a different diff than before.

Sometimes when you cherry-pick, you might not even hit a "true" conflict, but if you forgot to run "rerere", you might simply hit a previously solved conflict again.

pmeunier··on Commits are shapshots, not diffs
> Snapshots and diffs are different views of the same thing.

Not completely: think of conflict resolutions for example. In patch-based systems such as Darcs and Pijul, you don't need a "rerere" command to handle that special case.

And I'm sure you'll agree that conflicts is the one critical situation where we need the best possible tool.

> When I git push, I upload my commits (i.e. snapshots). And when I git cherry-pick, I apply a diff

That's not how Git does it. Instead, Git runs 3-way merge, which doesn't do what you expect, see https://pijul.org/manual/why_pijul.html.

pmeunier··on Commits are shapshots, not diffs
> I can't agree, because I don't work on diffs, I work on whatever the current state of the file hierarchy is. When I make changes to a file,…

This is somewhat contradictory. Of course, everybody works on states. But as you write yourself, you make changes to a file, you don't create a full new version from scratch every time you add a semicolon.

So, if you "make changes", you actually do "work on diffs".

> Further, diffs only make sense if you have the full version of the files, pre-diff. Without that full file state, diffs could operate in all sorts of incorrect ways.

That actually is an argument against Git: because Git doesn't model diffs, yet tries to simulate them, Git merges and rebases do all sort of incorrect things. For example, even the following simple example is not correctly handled in Git:

https://pijul.org/manual/why_pijul.html (see the figure "git merge vs pijul merge").

pmeunier··on Commits are shapshots, not diffs
> I'd argue that diffs in the presence of renames aren't natural, and git handles them better than other version control systems.

Maybe this is because I designed that part in Pijul, but I disagree. Pijul has renames commute with other types of edits, and renames are faithfully modeled in Pijul patches, not just "guessed" after the fact based on contents.

pmeunier··on Commits are shapshots, not diffs
> Patches are commutative (barring conflicts).

That isn't quite true, even without conflicts. The same example that shows that Git merge is not associative (https://pijul.org/manual/why_pijul.html) can be used to show that it also isn't commutative.

pmeunier··on Commits are shapshots, not diffs
I don't know about ponies, but for snapshots, I disagree: snapshots don't model diffs properly, and many operations in Git are essentially trying to simulate patches:

- Merge and rebase both try to "replay" patches. But because Git doesn't work with patches, it actually only simulates that, using heuristic algorithms such as 3-way merge. This works most of the time, but not always (see https://pijul.org/manual/why_pijul.html).

- Conflict resolutions in Git cannot be modeled as patches, and you need `git rerere` to simulate that. This is not anecdotal, since conflicts is the one situation where you need the best and most intuitive tool.

- Short-lived branches are another poor simulation of patches: if Git were really working on patches, you would be able to create a branch "after the fact", meaning that you would work (possibly on multiple features at the same time), and push different parts of your work separately, without having to worry about branches before you start.

pmeunier··on Commits are shapshots, not diffs
This is not inherent to the patch model, just to Darcs' algorithm. Pijul (pijul.org) doesn't have these problems, and is indeed quite fast. The main limitation at the moment is merging very large histories, which has the same complexity as a Git rebase at the moment (this will be fixed).
pmeunier··on Commits are shapshots, not diffs
That's right, and actually Pijul patches are not actual diffs, they just behave exactly like them for all uses.

The main thing I believe is intrinsically superior to Git (I use Pijul a lot) is patch commutation, not just patches. Pijul guarantees that two patches that could be written independently always commute.

This has important consequences:

- You can push patches from the same branch in any order (provided they don't depend on each other). This wont change their identity, meaning that if you decide to push the other patches later, you can do it without having to rebase, and review/test the new order.

- Conflicts happen between patches, and conflict resolutions are modeled as patches. This means that if you solve a conflict once, you can push your resolution, even if you have your own, private patches on the same branch.

pmeunier··on Commits are shapshots, not diffs
Actually, I wrote the graphical version of that example (the original authors are cited). I find that the code version is harder to understand. The letters make it much more explicit that Git is really shuffling lines around (I find that scary).
pmeunier··on Commits are shapshots, not diffs
You could certainly do that, and this is in fact one of the ways to use Pijul. However, you would completely miss out on the most important novelty of Pijul, which is patch commutation:

When we say "Pijul is patch-based", we don't only mean that it stores patches, but rather that patches that could be written independently commute, meaning that they can be applied in any order, and are guaranteed to give the same result in all cases (including when they conflict).

This simplifies many workflows.

pmeunier··on Commits are shapshots, not diffs
> What's that rebase/cherry-picking craziness?

I can only guess what OP had in mind: when you rebase in Git, commits are "replayed" on top of other commits. But there's no way this replay can really work, for two reasons:

- conflicts are not even modeled in commits, which is why some solve conflicts can come back. You may say this never happens, but (1) the problem is so real that Git even has a `git rerere` command to fix it, and (2) many workflows have explicit ways of avoiding this situation: no matter what your natural workflow is, you must adapt it to suit Git.

- each step of the replay tries to commute patches, but Git uses 3-way merge for that, and this is not a 100% algorithm, merely a heuristic, as explained there: https://pijul.org/manual/why_pijul.html

pmeunier··on Commits are shapshots, not diffs
They're not just diffs indeed, they have a bit more information, see an example there:

https://nest.pijul.com/pijul/pijul/changes/XTMYHJZLWWT5I2PJA...

First, there are explicit dependencies, and also each section starts with a somewhat cryptic machine-readable description, for example:

B:BD[2.1195] → [2.1195:1291]

One of the challenges in the new version of Pijul was to find a description that was printable in text, and not too-unintelligible to humans.

Darcs isn't alpha, but merges patches in exponential time, which was the initial motivation for Pijul.

pmeunier··on How to Make Your Code Reviewer Fall in Love with You
If you want a better example, see how I manage two different version of my SSH library for Rust, Thrussh: https://nest.pijul.com/pijul/thrussh

The problem this is solving is that the Rust ecosystem is in the middle of transitioning between two version of its async base library, Tokio: most libraries have transitioned, but the most used HTTP library (Hyper) hasn't. This forces any user that wants both HTTP and another network protocol (SSH, for instance) to stay on the older version.

However, we can't all keep waiting on Hyper. Pijul's solution is to push the same bugfixes to two different channels, one of which includes an extra patch for transitioning to Tokio-0.3 (the newer version). Since the identity of patches is preserved, no rebase is needed, and conflicts solved once are solved forever (no rerere).

pmeunier··on How to Make Your Code Reviewer Fall in Love with You
In the entire history of Pijul, you probably picked the best example of how to get your reviewer stop being in love with you. That change fixed several hundred errors caused by the removal of a dependency.
pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
> low visibility warnings

The versions were called 0.x, each blog post said "this is experimental", and there was even a blinking line on the front page of nest.pijul.com. Maybe we should have printed a notice on the command line tool itself ;-)

More seriously, one major issue with the 0.x versions was that bad performance meant we couldn't really test it massively. But now we can, and actually one of my first tests was to try and import the history of Nixpkgs, and run massive checks for data loss after every operation. It works now.

pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
> Then adding "Fix" to another channel will automatically pull in the "dependent change" "pre_Foo" as well, if applicable? Or is a separate "new_Fix" created?

I'm not 100% certain of which is which in your sentence, but if Fix and Fox are in conflict (maybe Fix is Alice's fix, and Fox is Bob's), and they are both pushed to the same channel, we need to solve the conflict. We do that by creating a change, called "Resol".

Now, if you push Fix to another channel (that doesn't have Fox), that doesn't necessarily pull in Fox and Resol. If you push Fix and Fox, you'll get the exact same conflict. But instead of solving it again, you can simply push Resol, and the conflict will be solved.

> Can a link between two changes be created manually, even though the delta does not require it? E.g. to attach an empty commit, if these are even possible in Pijul (compare git's "--allow-empty"). Or, more reasonably, to record logical dependencies which do not show up in the code, such a separate changes for testing vs. the code itself.

Absolutely, the syntax for that is, when presented with a change draft when running `pijul record`, add a line of the following form:

[] BQDE4VH6OZHULHAOP37GSBJMCI5QIFSOU6COBYKPAY6FPH3IMISAC

To the dependencies section (that is, empty square brackets, a space, and the hash of the change you want to add to the dependencies).

We can even imagine adding dependencies automatically with external tools via record-time hooks, taking the specific programming language into account.

pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
It does allow that, to some extent: by changing the diff algorithm. On thing you can do in a diff algorithm for Pijul (not written yet, but totally possible) is to treat whitespace as their own binary blocks, so that reformatting commutes with other changes.
pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
https://www.reddit.com/r/programming/comments/k39td1/pijul_t...
pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
> Please don’t get discouraged by comments pointing out the current limitations of the platform

Thanks for the kind message. I totally agree, and didn't take the comment as an offense at all (the reason I reminded the relative youth of Pijul is that I know from experience that some people expect Pijul to have many more users than it currently has).

pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
Yes! The `pijul git` command allows you to import Pijul repositories by replaying their history. It also works incrementally, meaning that if you already imported a repository, you'll be able to continue new commits.

This uses a somewhat naive way of doing things, and can take a very long time on large repositories. One other way of doing it would be to add an "initial change" explicitly saying "I'm coming from Git commit number #SHA1HASH".

pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
> As far as I understand, the main change with respect to Git is that commit hashes don't change depending on previous history, so if you rebase the same change IDs are retained.

That is not the main change, merely a by-product. There are many changes:

- Perhaps one of the most crucial differences is that Pijul uses a rigorous merge algorithm, and doesn't run into the bad merge examples described in [1]. Note that the situation described there happens without any conflict, and doesn't tell anything to the user.

- Changes that could be written independently commute, meaning that they can be pushed independently. You don't have to worry about isolating them on feature branches, which can save time if you (like me) work on many features at the same time.

- Rebase and Merge are the same operation in Pijul, and happens automatically. Conflicts happen between changes, and conflict resolutions are recorded as changes, so you don't need any `git rerere` (which essentially guesses what to do), since conflict resolutions work even in a different context.

- Everything is invertible! You can even remove an old change from a channel if you like, as long as other changes don't depend on it.

> I don't like that two repositories in different states could have the same commit ID.

That never happens in Pijul. Changes are not commits, they're diffs between commits. And states have identifiers too, they're just more subtle: if you have two changes A and B, Pijul can apply them in any order without changing the result (so why change the commit id?). In Pijul, if you run `pijul log --state`, on "A, then B" and "B, then A", you will get the same state id.

[1] https://pijul.org/manual/why_pijul.html

pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
We might not have read the same article.

> Also, the fact it is written in rust means nothing to me.

I'm one of the authors of Pijul, and the author of that post asked me a few questions when preparing that post, and that is addressed there: https://initialcommit.com/blog/pijul-creator (see "Why did you choose to write Pijul in Rust?").

> It doesn't explain the technical aspects, it doesn't allow me to understand the system

You can have plenty of that there: https://pijul.org/manual/theory.html.

> This was painful to read.

I strongly disagree. Explaining the goals of this project isn't always easy, because both large complex projects and complete beginners can benefit from using a rigorous mathematical modeling. I guess this blog post leans more towards the "beginners" side, whereas the "Theory" page linked above would be more appealing to power users. Both are useful and important, they just appeal to different people.

pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
About your guess on the Pijul hub: nest.pijul.com does something nicer than PRs, which is to attach changes independent from the context, to discussions.

See the following discussion, for example: https://nest.pijul.com/pijul/pijul/discussions/104

These two contributors (@cole-h and @lowenheim) are trying to add colours to the output of `pijul change`, and discuss changes, rewrite them, amend them, push them again. Certainly, some of these changes conflict with others. But that's totally ok, since you can pull them independently.

For example, when preparing release 1.0.0-alpha.9, I didn't pull the latest patch from that discussion into my main local channel, and I preferred to pull some of the older ones instead. I did this because the latest one doesn't detect whether the standard output is a TTY. If I apply them to the main channel on the Nest, my current local channel will be in sync with it, even though I've worked on ten other features since then.

pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
This is incredibly useful for a project like Nixpkgs.
pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
> I'd especially be interested in examples regarding multiple channels / branches (one development and several production branches), and then backporting a fix into older versions.

I've been maintaining to versions of my SSH library for Rust, called Thrussh (https://nest.pijul.com/pijul/thrussh).

For some context, the Rust asynchronous ecosystem is undergoing a change, with a fundamental library called Tokio going from version 0.2 to 0.3. Some crates have made the change, others haven't. So if you're like me, and some of your code implements both an SSH server and an HTTP server on the same event loop, you might want to stay with Tokio 0.2 (since Hyper, the main HTTP library, hasn't published the move yet), while allowing others to benefit from all the bugfixes you find, and start transitioning now.

The cool thing about doing this in Pijul is, you can work on your Tokio-0.2 version, and push your changes directly to your Tokio-0.3 version without having to care about cherry-picks, merges and rebases explicitly. Do this locally, test, publish.

> How is the workflow if this applies cleanly to rel.22, but not rel.21 anymore? What if the same conflict happens in rel.20, rel.19 and rel.18?

Good question. In Pijul, conflicts occur between changes (not between commits or states), and are solved by changes. This is in contrast with e.g. `git rerere`, since the solution to a conflict depends only on the changes that introduced it, not on anything else. This means that once you've solved the conflict in rel.21, you can push it to rel.20, rel.19 (it's the exact same change, with the same hash).

> What if the conflict resolution changes every single line, but I still want to record that, "Yes, this fix is also in rel.17, even though it does not look like it".

Excellent question. Two answers:

- If you're really doing this, then your intention cannot really be guessed from your change, which sort of defeats the point of using changes, and you will get another conflict. If you fix that conflict again though, that fix will apply to any "branch" (Pijul users call these "channels" because they are not exactly the same as Git branches) that has the two changes.

- There is ongoing research to extend the theory to allow large restructuring of code to commute with edits to that code. Note that no other system handles that well: Git and Mercurial don't even consider commutation (but their merge heuristics handle refactoring correctly sometimes), Darcs doesn't do it at all. Since our goal is to get something that works 100% of the time, we have to be a bit more careful.

pmeunier··on Pijul – The Mathematically Sound Version Control System Written in Rust
> Firstly, there are basically no potential colaborators for the areas i work on

That's correct. On the other hand, this is a network effect, and the Nest, in its current incarnation, is only 3 weeks old. It's getting better every day, though.

The two main features I'm working on at the moment are:

- CI/CD. This is almost ready to go, it is "sort of" working. One issue that I fixed recently is that Pijul versions are tricky to compute: change A, followed by B, is the same state (guaranteed by design) as B, followed by A. Our version identifiers handle that now (try `pijul log --state`).

- Social features, like discoverability, recommendation of collaborations. We're really just getting started at this, and figuring out what works best.

I should add that until two weeks ago, the Nest would stop accepting connections at random times, or crash. It's much more stable now, and we can finally think of adding features.

> Secondly, it is much less mature ass a platform

There are tags for issues. You can configure them in the "Admin" page of your repositories. They're not used very much for Pijul itself, because most issues so far have been closed within minutes or hours after their creation.

> Which is a mind twise of an idea. But they say they solved it, so nice.

As one of the authors, I can tell you it wasn't easy to get everything to work together, especially for conflicts. But the theory is clear and sound now (and I have good confidence that the implementation is, too): any two patches that could have been created independently can be pushed to a remote repository independently from each other.

pmeunier··on Pijul: Towards 1.0
Alright, maybe you should have read my answer before replying. I do know what I'm talking about, I wrote the damn thing.
← PreviousPage 6 of 10Next →