HNHacker News
TopNewBestAskShowJobs

pmeunier

672 karma · joined July 13, 2014

submissionscomments
pmeunier··on Pijul 1.0 Beta
The linked shown above is the erroneous one, the correct one is https://pijul.org/posts/2021-02-06-rethinking-sanakirja/
pmeunier··on Pijul 1.0 Beta
It is actually better documented, the design is explained in the code: https://docs.rs/sanakirja-core

See for instance https://docs.rs/sanakirja-core/latest/sanakirja_core/btree/p...

Sanakirja even reached a point where bugs were found only in the last few functions without comments. Writing the comments to explain exactly what each line does in all cases (there are lots of cases everywhere) fixed everything.

pmeunier··on Pijul 1.0 Beta
Indeed. I just fixed it.
pmeunier··on Show HN: The Best of Git, Mercurial, and Pijul in One VCS?
One thing is somewhat subjective: liking a feature or not.

However, if you're claiming to make a system including "the best" of these three tools, any reader will expect the obvious property that Jujutsu can faithfully simulate all possible workflows from these tools.

Since it is essentially a layer on top of Git, I am fully convinced that it can simulate Git (for example by using Git directly if needed). But since you fail to explain your algorithms and core ideas in the docs, I'm also convinced that it can't simulate draft changesets from Mercurial.

Since I know what it takes to simulate patch commutation (from Pijul), especially together with the snapshot-based nature of Git, I also doubt that it can simulate that. Obviously, I'd be happy to be proven wrong, because I'd learn something really cool!

I've asked you this question before somewhere else: I still don't know whether Jujutsu satisfies the most basic property any user expects from merges: associativity. If not, it's ok, Git, Mercurial, SVN, Fossil also fail to guarantee it on all merges.

I'm also surprised you aren't claiming to do better than Darcs. Even though I've never used `darcs replace`, and I failed to implement that command in Pijul, I'm nevertheless a bit jealous, because I find that quite elegant.

pmeunier··on Show HN: The Best of Git, Mercurial, and Pijul in One VCS?
Serious misunderstanding of Pijul there, Jujutsu seems more similar to Gitless. If you don't have patch commutation, or if you can't explain why your merge algorithm is more than a heuristic (in your case, both), you can't include "the best of" Pijul.
pmeunier··on Where is the CRDT for syntax trees
> You didn’t mention the lack of a good theory for tree diffs/patches here. Is that because you think there is a good theory or because you think not having a theory doesn’t matter? Or just that it wasn’t worth mentioning?

I'm convinced there is a good theory. Moreover, since line graphs (i.e. totally ordered bytes in a file) are just a particular case of trees, that theory could even be implemented in the same way Pijul is now, where the "easy" case just means blobs linked together.

But I'm also convinced that there are so many edge cases that I wouldn't want to even start considering that before Pijul sees some real-world usage at a decent scale. If everybody misses the point and ends up preferring Git, then why even bother?

> though I don’t see that behaviour with operating systems. Perhaps they are considered too sacred or something.

I guess everybody sees how they would start to write a VCS: just read files, maybe write diff, imagine some datastructures to store that, fix the bugs and problems, that's it. Fewer people can see how to write operating systems, and even people who follow hobby tutorials immediately realise the complexity of handling all the hardware.

> I think most programmers aren’t stressing enough file systems enough to find that they are sometimes buggy and hard to deal with accurately and efficiently, and if pressed might mumble something about calling fsync.

That's right, but on the other hand if you look at the "massively parallel, structured filesystems", called databases, the space is tiny:

- There's a multi-billion-dollars company (Oracle) built at a time where nobody wanted to write fast databases.

- An academic project (Postgres) which grew extraordinarily slowly, and somehow managed to survive as a research platform, funded by public funds.

- Two single-author projects led by extraordinarily motivated people, one of which isn't meant to support concurrency (SQLite), and MySQL/MariaDB, built because there was no affordable, non-experimental database solution.

- Recent years have seen some progress funded by giant companies like Google for their cloud infrastructure, but we're not yet seeing many CRDTs in production.

The same goes for theory-based programming languages: both Haskell and OCaml have barely survived for 20 years before being used at massive scales. Rust has had a slightly easier time, but not much easier, despite being supported by Mozilla.

Yet, most engineers would consider databases "basic" technology, because they use them a lot, and (quite fortunately) don't need to implement them to build on top of them.

pmeunier··on Where is the CRDT for syntax trees
I've thought a little bit about those when working on Pijul, there are multiple issues, mostly at the interface between the technical and the human:

- There is no good diff for trees. Some formulations of the problem are NP-complete, meaning that patches won't be minimal, and hence users will get unexpected conflicts.

- Most VCS users have learned to be afraid of their favourite tool, and treat it like something extremely fragile that can break at any time for unknown reasons. Words like "porcelain" show that feeling is even present in the tool's authors themselves. For that reason, the more layers and heuristics one adds on top of basic tools, the scarier it becomes (especially for experienced users).

Writing a VCS is hard and takes way more time than most people expect. Few applications share the stateful nature of VCSs, one needs to feel the pain of designing and implementing databases and/or filesystems in order to understand these systems, where any bug can mean the state becomes corrupt, data is lost, etc. which is the very thing the system you're writing is meant to prevent. So, that kind of effort will only make sense when there are enough users to justify it and support the development, in terms of both man-hours and money.

pmeunier··on Highlights from Git 2.34
That could work. If your local patches are complex enough you could even rearrange them using patch commutation.

At the moment, Pijul is in a state where there is just one remaining feature I want to implement before beta, with a massive expected impact on disk space (and a limited impact on speed as well).

For the kind of "smaller (but essential) features" like what you're suggestion, there are tons of those, and we can totally implement them after beta. Git export is a common request we get, though. I'm just scared of how people will react when they export and then reimport their patches elsewhere, and get tons of fake conflicts.

pmeunier··on Highlights from Git 2.34
First, you can easily import Git repos into Pijul.

One issue with two-way interop is that while importing a commit into a patch is easy, doing the opposite is more complicated. Indeed, since Pijul is patch-based, there is no way to map patches onto commits in a unique way, since a single patch is quite independent from the context (other patches around it), which is what makes Pijul easy to use.

Also, if Alice and Bob work on a common Pijul server in parallel with their Git server, they'll get artificial conflicts in Pijul, even though they have the same contents.

pmeunier··on Why SQLite does not use Git (2018)
First, congrats for replying to a comment asking you for concrete arguments without using any of those, nor replying directly to anything that was said.

Then, I don't think you actually understand what we mean by patches: for anyone who has looked into the topic, patches are obviously not equivalent to commits except in the most basic cases (where all tools "work", in your own terms).

A datastructure is defined only with respect to the operations you define on it, I'm sure you use other commands than checkout. For example, people who review snapshots rather than patches before a merge (or even a commit!) either have really small projects, or way too many man-hours to spend.

> if you take one or two hours to learn the documentation is actually intuitive

Git isn't "intuitive" at all for anything but the most basic operations (I agree it is easy for those, if you are willing to work with snapshots). For example, the existence of `git rerere` clearly shows that Git doesn't model conflicts at all. Then, merges are not unique, Git chooses one at random when there are more than one. Rebase is a horrible hack, "replaying" commits doesn't even always work.

Moreover, saying Git is easy isn't a very effective way to show off on HN: if you feel you have to defend Git so strongly, even though it wasn't actually attacked, and to the point of refusing to reply directly to any rational argument, it clearly shows you invested enough time in Git to feel you would lose something if you had to move to a better solution. So, sorry, but I don't think it was just "one or two hours" for you.

pmeunier··on Why SQLite does not use Git (2018)
An easy dismissal without any solid arguments. First, saying "Darcs is slow and Pijul…" is strange, since Darcs and Pijul aim at solving the same problems.

Then, Darcs has never been slow for me, I've been using it for about 15 years, I can remember of one minor performance issue. It doesn't scale well to very large projects, but there are extremely few of them. Since concrete arguments are important when discussing with others, the main event which started Darcs' decline was its inability to handle conflicts in GHC (the Haskell compiler), which was at the time the open source project with the largest history (as measured in number of patches).

As for Pijul, I'm curious where you got your information from. I know Pijul's authors well, at least enough to know that they are well aware of the algorithms used in Git. Pijul does solve a number of problems:

- I know at least one very large-scale project where collaborators are well aware that 3-way merge can mess with their code in really bad ways, and complain about wasting tons of time with that. Unfortunately, I can't embarrass the project's authors by sharing their name, but I can say that it is a widely used piece of software with a really long history.

- Knowing that the code you review is the code you merge is the only thing you should care about if you care about your code at all. This is not what 3-way merge gives you, as the Pijul manual clearly explains (see "lack of associativity").

- Patch commutation is what Git tries to do all the time (rebase, merge, rerere…), except with bad hacks which fail in the more complex situations, which is where we need them the most.

pmeunier··on Ask HN: Alternatives to Git – what are you using?
It is absolutely not a rewrite of Darcs, the main difference being that Darcs works with transforms only, whereas Pijul has points between the transforms, allowing it to scale to much larger repos.
pmeunier··on Visual Git
3-way merges suffer from two problems:

- They are not "associative", as mathematicians say, which means that if a remote branch has two commits A, followed by B, merging A then B might give you a different result from merging just B. Stockholm syndrome has convinced many programmers that this isn't a problem, but it actually prevents the very sane reviewing process of reading individual proposed changes along a branch. In other words, this is one of the main reasons Git users have to follow "flows".

- Conflicts! 3-way merge treats conflicts as a failure to merge, and tools based on 3-way merge (Git, SVN, Mercurial, Fossil, CVS…) stop everything when a conflict happens. Now, remember that these tools are based on snapshots, and snapshots can never have conflicts. Therefore, conflict resolutions are forgotten, the only thing remembered is that "Commit [insert SHA1-hash] (Revision number [insert integer] in SVN) comes after these two parents". This is so wrong that Git even has a command to "pseudo-fix" it, called `git rerere`. Again, if this doesn't sound like a problem to you, this could be because Stockholm syndrome taught you that the tool gets upset if you merge from the same remote branch twice, or if you fork a branch that isn't called "master" and isn't on GitHub. In other words, because you're actually using a centralised version control tool ;-)

pmeunier··on Visual Git
> (and source control systems you don't hear much about these days like darcs/pijul)

A bit unfair, Darcs is indeed less fashionable nowadays because of performance issues on very large instances (which few projects actually have), Pijul solves these problems but is still alpha. In other words, this lineage of version control is between two different tools.

pmeunier··on Ask HN: Alternatives to Git – what are you using?
> someone taking a snapshot-happy copy-on-write filesystem (btrfs, nilfs2), and putting a version control layer on top.

That's actually what Pijul does under the hood, its CoW filesystem is called Sanakirja (Sanakirja is a bit more than just a filesystem).

(I'm one of the authors).

pmeunier··on Pijul – Datacenter on Fires
Forgot to say: thanks! Staying on-topic in a post-mortem is hard, there are so many things involved…
pmeunier··on Pijul – Datacenter on Fires
First, let me restate something I wrote on the post: I want my service to be free from politics, and I was ready to consider any hosting service that I could trust would allow me to do that.

The word "partisan" is a bit ambiguous in your comment: just to clarify my position, disagreeing with one side (however strongly!) doesn't mean supporting the others.

The reason I wrote that lengthy argument against choosing Scaleway for this project, is that Scaleway was in a grey area for me: while it is obvious that the other two providers I mentioned aren't a good fit for this particular project, Scaleway is extremely competitive in many regards, and I've had to go through a lengthy reasoning to imagine what could go wrong for this particular project.

That said, I can see how a lengthy reasoning shouldn't always imply a lengthy phrasing, and in fact I agree that the statement in the post seems to mean the opposite of what I meant: since I wrote many more words about Scaleway, it does seem like I was trying to be mean to them, which is quite the opposite of what I really meant.

So, thanks for making me see that, I've fixed it in the post.

> but saying they "organized a militia against its fellow citizens" is about as honest as saying "the US government is running extermination camps" because it detains immigrants.

Detaining migrants is not exterminating them. Private people paid by to go beat up random protesters are usually called militiae. Moreover, Valeurs Actuelles is quite different from Fox News: in particular, Fox News is not explicitly promoting the slavery of black people, nor promoting military coups against the US government.

My only point here is: I believe people should be able to express and discuss any opinion that doesn't contradict my freedom to disagree. Note that in my (now amended) blog post, I didn't mention anything "political" in the sense of "open to debate".

> Scaleway is out of bounds because its largest owner is a supporter of the President of the Republic

Sorry, but you got that completely wrong. I obviously failed to communicate an essential difference I see between what I call politics (i.e. debatable opinions) and the "other thing", which is a denial of the basic right of others to disagree.

My point in the post was, someone who buys press outlets might not hesitate to do the same for contents they dislike on web services they host, or at least there is no strong indication that they won't do that. Given the fact that some cloud providers have done it in other European countries (as stated in the blog post), I wouldn't be surprised myself.

But even more strongly, I don't understand what to expect from that "other thing", and that scares me.

> What's next, you're not allowed to connect to a Pijul database if your ISP is Free

Again completely wrong (ironically, my mobile phone is on Free, and I've never used another ISP at home): the perspective from an ISP point of view is different from the hosting point of view. If one ISP decides to censor a website, this doesn't hurt the website very much, since users can change at a very low cost. This doesn't apply for cloud hosting, where the cost of moving can be really high.

> You're not allowed to contribute to the code if you were taught at the 42 School? (yes, I'm caricaturing, but still)

How is that even remotely connected to the issue of deciding where to host a web service?

> But advertising "we won't work with this company because a shareholder is a support of the president and the president went on French Fox News once" is the kind of ultra-divisive

It is actually the opposite of divisive: I want my service to be usable by people regardless of their political opinions, and this is only compatible with basic values of respect, tolerance and free speech. What I'm talking about here is even weaker than free speech, it's only the right to disagree without being humiliated (caricatured as a slave) or beaten up.

pmeunier··on Pijul – Datacenter on Fires
> I wouldn't discount your work, etcd is very stable, so it probably works very well for your use case.

I had understood that, but code I don't write is code I don't have to maintain, so I'm obviously always totally open to suggestions (getting comments and suggestions is part of the reason I wrote that blog post to begin with).

pmeunier··on Pijul – Datacenter on Fires
Staying at OVH, with Cloudflare in front of the servers.
pmeunier··on Pijul – Datacenter on Fires
> do I understand that all database traffic gets routed to the leader? Do you serve read-only queries from the local replica?

No, and yes. The replica have a very large ping between them, serving everything from the leader would be extremely slow.

> The described setup looks like something that cockroachdb would be able to handle nicely, which perhaps is a little more battle-tested than what you wrote.

Thanks for the suggestion, I wasn't aware of Cockroachdb, I'll try to see how I can switch to that.

Also, "what I wrote" is just a small script to extend my previous setup (which used PostgreSQL) to replicae. I first read about Patroni, supposedly used by large companies, and went with that, before discovering that it requires a lot of monitoring, and can't work for a one-person team.

pmeunier··on Pijul – Datacenter on Fires
As noted on the blog post, it's also instantly mirrored to the cloud with Restic, in addition to being replicated multiple times live. The i3 notification is meant to keep a constant eye on the thing, to make sure it does happen every day.
pmeunier··on Pijul – Datacenter on Fires
Thanks, that's actually why I wrote "European" rather than "in Europe", both in the blog post and in my previous comment.

I'm glad to know that you agree with that distinction.

pmeunier··on Pijul – Datacenter on Fires
OVH is the largest European cloud company. Amazon, Google and Microsoft don't count as European.

As a user of both, the price different may not be 75x, but I wouldn't be surprised if it were in the 10x-20x range. The polish is also quite different, and is consistent with the price difference.

pmeunier··on Commits are snapshots not diffs (2020)
> In general, the patch 'patch1 ∘ patch2' can be different from the patch 'patch2 ∘ patch1'.

That isn't true. In Pijul, either patch1 explicitly depends on patch2, or patch2 explicitly depends on patch1, or else these two things you said are equal.

pmeunier··on Commits are snapshots not diffs (2020)
It has made huge progress lately, which was always the plan: the new data representation introduced in November 2020 made it scale to huge repositories (this was impossible before, because of disk space and speed), and then we also changed the backend recently (https://pijul.org/posts/2021-02-06-rethinking-sanakirja/).
pmeunier··on Commits are snapshots not diffs (2020)
I suggest you read more about Pijul and Darcs, because you seem to be confusing Pijul patches with the output of `diff`.

Patches are much easier to work with, more reliable and fully deterministic. For example, merge and rebase are the same operation in Pijul, you can remove an old patch without changing the identity of all the patches after it, and yet have strong version identifiers, with the exact benefits you describe for snapshots.

pmeunier··on Commits are snapshots not diffs (2020)
I'm one of the authors. The concepts are different, we do have a concept called "tags" (not ready yet) which is more efficient than shallow clones, in the sense that you can make patches (commits) on top of a shallow clone without any down side (unlike in Git, where shallow repositories can become slower).

You can also do partial clones in Pijul: since patches commute, the patches you produce on top of a partial clone can be pushed to the full repository in the exact same way.

pmeunier··on Commits are snapshots not diffs (2020)
There's zero theory on the following page:

https://pijul.org/manual/why_pijul.html

There are many answers:

1. Commutation makes your life easier because you can be much less careful about how you manage your branches. Rebase, merge and commit are the same operation (apply a patch), without any loss of power: you can simulate all of Git within Pijul, except for bad (i.e. silent non-associative merges), which Pijul doesn't have.

2. Everything is easily reversible. I know all actions in Git are reversible in some way, but not in the same way: for example, you can't undo an old patch without changing the identity of all the patches after it. I know you're thinking this is important for strong version identifiers, but Pijul also has strong version identifiers, just without the compromise on usability. This is achieved using cool cryptography tricks.

3. Solving a conflict in Pijul actually solves it. Conflicts happen between two (or more) patches, and are solved by a patch: if the same two patches are on another branch, you are guaranteed to get the exact same conflict in 100% of cases, and that conflict is solved by the very same patch that solved it in the first branch.

4. When merging, Git solves an optimisation problem that may have multiple solutions in some cases. Git chooses one arbitrary solution based on the content of the lines, and doesn't warn you if there are others (because that would be a very hard computational problem). Pijul doesn't do that, and gives you strong guarantees on merges. You still have to test, but when reviewing, you can predict in your head, with 100% accuracy, how Pijul will merge. This isn't the case in Git: lines inserted at the end of a file might be merged into unrelated lines at the beginning of a file sometimes, if Git feels like it.

pmeunier··on Commits are snapshots not diffs (2020)
To the contrary, that example actually gives a solid argument for three points where Pijul is better than Git:

1. The order between lines is preserved by Pijul. This is important: let's say Alice works at the beginning of the file (lines 1-10 of 1000), whereas Bob works at the end (lines 990-1000 of 1000). Pijul preserves the order in all cases, whereas Git might randomly decide, based on the contents of the lines, to merge Bob's new lines in the middle of Alice's new lines.

2. Git solves an optimisation problem (3-way merge) that may have multiple solutions. Unfortunately, there is no way to count the number of solutions, or even to tell whether there are multiple solutions, in a reasonable amount of time. Git therefore picks one solution silently, based on the contents of the lines. In contrast to that, Pijul is deterministic.

3. Pijul is associative, meaning that merging A and (BC) is the same as first merging A and B, and then merging C; in other words, you can merge a branch commit by commit. Git doesn't have that property: if you merge a branch, you MUST (1) stop working on it, or else the future merges become totally unpredictable, and artificial conflicts might come back (yes, I know about dirty-hacks-to-try-and-fix-that-when-they-work such as rerere), and (2) check the result of the merge extremely carefully, because in addition to the logical errors that merges can reasonably introduce, Git might also introduce extra unpredictable errors by randomly shuffling lines around.

pmeunier··on A Categorical Theory of Patches (2013)
Pijul isn't really an "implementation" of that, for many reasons, including the fact that the paper doesn't show an algorithm to implement.

However, Pijul was inspired by this, with a number of differences, including support for line deletions (which changes everything) and multiple files, which cause many more conflict types.

Darcs was another major source of inspiration.

← PreviousPage 4 of 10Next →