HNHacker News
TopNewBestAskShowJobs

pmeunier

672 karma · joined July 13, 2014

submissionscomments
pmeunier··on The Source Code Control System (1975) [pdf]
I'm not familiar with that thesis. Pijul uses deltas everywhere (but a special form of deltas, storing more information than the output of `diff`), plus tricks to make that fast.
pmeunier··on The Source Code Control System (1975) [pdf]
> It strikes me that I treat the result of git-rerere as a ridiculously weak version of this

I'm the main author of Pijul, and rerere has been my main argument in every discussion about Pijul to explain how everybody is trying to simulate commutation instead of really achieving it.

This is the first time I hear it from somebody else. THANKS!

Also, Darcs' theory doesn't really work around conflicts. Also, until recently, applying a patch in Darcs could take a time exponential in the size of history. It seems to be quadratic in the last few versions (Pijul is in log of the size of history).

pmeunier··on Little is a statically typed, C-like scripting language
I came back here to say:

> I've built two commercially successful systems,

This is a great achievements, congrats on that!

All I was saying in my previous comment was, these very cool achievements don't prevent you from calling all other people "idiotic", and from saying "they don't have a clue" without even looking at their designs, or even thinking that they might have had ideas different from yours.

Also, calling Git "patch-based" is quite wrong, but my point wasn't technical.

pmeunier··on Little is a statically typed, C-like scripting language
> The comment that RCS scales for binaries couldn't be more wrong, RCS hates binaries with a passion.

That isn't what I meant, I was actually talking about Perforce. RCS doesn't even handle multiple files.

pmeunier··on Little is a statically typed, C-like scripting language
(Also, our Discourse and Zulip have been pretty civil places until now, I'd like to keep it that way).
pmeunier··on Little is a statically typed, C-like scripting language
Turns out I also hang out here sometimes ;-)

I was about to reply point by point with arguments, but I changed my mind:

I don't think I'd be interested in that conversation, the comment above is worse than a public display of a very poor understanding of Git and Pijul: it also shows a complete ignorance of other actors in the market. It turns out the market leader for big repositories, Perforce, is itself based on RCS, possibly (but I don't know for sure, since Perforce is proprietary) because RCS scales much better to large binary assets than other solutions (I'd argue Pijul solves that, but that's beside my point).

I am a big fan of Git, Mercurial and Darcs myself, my co-author on Pijul was actually a maintainer of Darcs for many years, and I'm actively collaborating with the maintainers of Mercurial at the moment. And even though Perforce and Plastic are closed-source, they do solve one problem (scalability) which distributed systems are only beginning to understand (if there's one thing I think we've achieved with Pijul, it is about getting beyond the "distributed vs. scalable" trade-off).

Here's my take on the comment above: I don't think you can build good systems without understanding how others are made. There are no free lunches, no silver bullets, no geniuses. Only good ideas, bibliography, and hard work.

pmeunier··on Rethinking Sanakirja
> Thanks to the author for an enlightening article.

Thanks, I'm glad it was useful!

pmeunier··on Rethinking Sanakirja
Thanks! My understanding was that even these grammatical similarities (between Hungarian and Finnish) could be somewhat coincidental.
pmeunier··on Rethinking Sanakirja
Well, I must admit that users currently need to know a little bit about how things work in order to use it, it isn't completely foolproof.

As an example, the type of channels/branches in Pijul contains things equivalent to `BTreeMap<String, (BTreeMap<Vertex, Vec<Edge>>, BTreeMap<u64, Change>)>`, and failing to update the pointers can result in corruption. Of course one can provide a safe interface on top of that (which I do in Pijul), but that depends on the types, so it isn't done yet (macros to derive safe implementations would be a really cool project btw).

Edit: to answer your question more specifically, small sectors might make it silently non-atomic, but there are workarounds (for example by using an extra level of indirection).

pmeunier··on Rethinking Sanakirja
The "Finno-Hugric" family is a bit controversial. That said, I speak several indo-european languages, and Farsi still sounds very exotic to me.
pmeunier··on Rethinking Sanakirja
1. Correct, my point was to emphasise that the implementation is the same. I've just updated it, thanks!

2. We often make the assumption that pages can be flushed to disk atomically, which is not very far from true on modern disks. Because the only update that really needs to be atomic is for these tables, they're all stored on one page. Since these pointers are all 64 bits in size (since 4k = 2^12, 12 bits in each of these are 0s), after a header of a few bytes, that leaves about 500 positions. The first few are probably more often atomic than the last ones, especially on very old disks.

3. It's still very important, but less so, because the Pijul repositories are much lighter on disk. Copying 1Mb is fast, but copying 100Mb is 100 times slower. I still like this feature, because (1) it costs almost nothing extra and (2) I really like the implementation.

pmeunier··on Rethinking Sanakirja
Good point. As this is a technical post following up on discussions in the Rust community (on Reddit and IRL in France), I forgot to add that word of background (I've fixed this now, thanks!).
pmeunier··on Implementing cross-process locks
What we have to do in Pijul is even worse: in addition to all this, it must (1) be cross-platform and (2) avoid any user interaction, because Pijul is meant to be beginner-friendly.
pmeunier··on Some Git internals
> Is pijul’s on-disk format stable yet?

Probably. The patch format is very unlikely to change. The repository format may change a little bit still.

I'd say it's probably ok to try and learn it now, but you should maybe wait for a few weeks before using it for something serious. On the other hand, we use it for itself, and I use it personally for most of my projects.

pmeunier··on Some Git internals
Applying a patch in Darcs usued to be in O(2^n), and now apparently O(n^2), where n is the size of history.

Applying a patch in Pijul is in O(p c log n), where p is the size of the patch and c the size of the largest "deletion-insertion conflict" p is involved in, where a "deletion-insertion conflict" is a situation where Alice deletes a block of text while Bob adds stuff in that same block.

Note that this is a rough bound, since all non-conflicting operations in a patch are in O(log n), except those involved in a "deletion-insertion conflict", which are in O(c log n).

So, Pijul is in fact faster than Git for merging (and rebasing). The only tradeoff at the moment is that going arbitrarily far back in history isn't as fast as it could be (this will be fixed very soon).

pmeunier··on What comes after Git?
Well, the actual complexity of apply and unrecord is O(p c log h), where p is the size of the patch, c the size of the largest conflict in which p is involved, and h is the size of history. That makes Pijul perfectly usable, even on large repositories.
pmeunier··on Ask HN: How Stable Is Pijul?
Thanks! It is a little bit challenging indeed. I've worked on worse systems before, and as always with asynchronicity, the devil is in the details, and there are more details than anyone who hasn't written it can imagine ;-)

I don't expect the semantics to change much. The things that need polishing are probably on documentation, error messages, and network. The latest bugs in Pijul itself (i.e. not Libpijul) included handling the case where your HTTP connection drops while you're downloading, and keeping your connection busy, that sort of stuff.

pmeunier··on Ask HN: How Stable Is Pijul?
By the way, when I wrote "code refactoring", I didn't mean "refactor Pijul", but rather supporting the following scenario:

- Alice splits a file into multiple smaller files. - In parallel, Bob changes one function.

At the moment, when they merge, they will get a conflict (both the same conflict, obviously), but I think it could be possible to remove the conflict.

pmeunier··on Ask HN: How Stable Is Pijul?
I'm the lead developer of Pijul. The theory is 99% there. Things are still moving a little bit, but it's mostly cleanup at this stage, the maths are solid. The UI is also moving, as users discover "weirdnesses" (which most of the time are places where Pijul's defaults differ from Git's).

There are two things I'm working on at the moment, and when these are done I believe 1.0 will be super close:

- Future-proofing the formats. There is one major thing I'd like to address before calling it 1.0, and that's code refactoring. It isn't supported at the moment, and Pijul is perfectly usable without that, but I'd like to keep the possibility of implementing it in the future open. I believe the current formats allow that, but I still need to be 100% certain.

- Merging two giant histories takes a time linear in the size of the history on the remote repository. Of course this isn't an operation you want to do very often, but it may become problematic for truly gigantic repositories, and I know how to fix that. This is planned to be a separate project at the moment, but it does require some support from the libpijul API.

pmeunier··on Pijul: Commutation and Scalability
Not yet, but this is just by lack of time for implementing the diff algorithm. There's nothing hard there, but there are other priorities at the moment.
pmeunier··on Pijul: Commutation and Scalability
On the other hand, it's still in alpha. What version did you try?

Also, pijul log still depends on less, and there are some problems related to that. This is going to be fixed soon.

pmeunier··on Pijul: Commutation and Scalability
> Is it possibly to lazy fetch the repository so you can keep H as low as possible? Like only look at H that are in conflict or something?

Something like that is announced at the end of the post ;-) It will hopefully be possible within a few days. The design is mostly done (at least for the basic feature), but my solution changes the implementation quite a bit, so I would expect temporary breakages (we're still in alpha, after all!).

pmeunier··on Pijul: Commutation and Scalability
Yes it does, because Pijul has an internal model of conflicts. In particular, conflicts happen between two patches, and are solved by a patch.

This means for example that when you solve a conflict on a branch, and pull from the same remote, the conflict won't reappear (no `git rerere` needed).

Or that you can push a conflict resolution to a remote, even if that remote is in another state: as long as the remote has the conflicting patches, the resolution will work.

pmeunier··on Pijul: Commutation and Scalability
There are global states in Pijul, using cryptographic primitives to handle the commutative nature of Pijul history.

I explained it in a previous blog post: https://pijul.org/posts/2020-11-07-towards-1.0/#version-iden...

pmeunier··on Pijul: Commutation and Scalability
I fixed that on the site, thanks.

About recent features, I used to be very careful about using a stable Rust that was at least a few weeks old, or one generation behind. But the latest release got an unexpected number of contributors to the project, and we didn't even have a CI system!

We do have a basic CI now (https://nest.pijul.com/pijul/pijul/ci), so that's one thing we definitely want to check.

pmeunier··on Pijul: Commutation and Scalability
There are two separate things:

- Git does have spurious conflicts coming back for no reason. The problem is so real that there's even a `rerere` command to take care of it.

- Pijul has "minimal" conflicts, in the sense that these are conflicts you can't possibly make sense of without human guidance. You get the guarantee that fixed conflicts don't come back. Unfortunately, "obvious" cases are never completely obvious: let's say Alice and Bob both add a case to a C function, and both change `#define NUMBER_OF_CASES 42` to `#define NUMBER_OF_CASES 43`.

pmeunier··on Pijul: Commutation and Scalability
Did you try the following:

cargo install pijul --version 1.0.0-alpha.24

Difficulties to compile have been reported on Windows, we're still trying to figure out a Windows CI pipeline.

pmeunier··on Pijul: Commutation and Scalability
It is actually defined as "you get the same file contents and directory structure, regardless of the order in which you apply the changes", which is quite good.

> But, this seems to require semantic level understanding of the code

No: you may very well have two different orders yielding the same output, even though the output is incorrect.

> This method seems to rely inherently on the fact that git commits are ordered. How would this work with a pijul-based flow?

Good question. Our CI over at https://nest.pijul.com/pijul/pijul/ci works in the same way. Patches are ordered in Pijul too, but the order is local. In your example, if Alice wins, Bob will pull her patch, fix his code, which will produce another patch. He will then send it to CI again (or directly to main if he's careless), and Alice will pull both Bob's original patch and the fix.

In the end, they will have different histories, but the exact same contents.

pmeunier··on Pijul: Commutation and Scalability
Pijul will treat that as a conflict indeed, in particular because it has no way to tell that Alice and Bob meant the same thing when adding that file. A README is a simple example, but what about a module split from a larger module during a refactoring? Even if the filenames added by Alice and Bob match, how can we know they're meant to be the same?
pmeunier··on Commits are shapshots, not diffs
All the arguments you give are based on shortcomings of the tools: Microsoft Word doesn't have version control, Wikipedia stores versions in a database.

> But the ultimate goal is to produce a version of the file.

The fact that the end result is a version doesn't mean versions should be the main object. If you add a semicolon to a file, the actions you do are "press the ; key", not "create an entire version from scratch that has a semicolon there".

> If you want a diff management system, play around with quilt

You're doing it again! Why pick the least suitable tools for the jobs? Quilt is not a patch-based version control system, Pijul and Darcs are.

← PreviousPage 5 of 10Next →