672 karma · joined July 13, 2014
I'm the main author of Pijul, and rerere has been my main argument in every discussion about Pijul to explain how everybody is trying to simulate commutation instead of really achieving it.
This is the first time I hear it from somebody else. THANKS!
Also, Darcs' theory doesn't really work around conflicts. Also, until recently, applying a patch in Darcs could take a time exponential in the size of history. It seems to be quadratic in the last few versions (Pijul is in log of the size of history).
> I've built two commercially successful systems,
This is a great achievements, congrats on that!
All I was saying in my previous comment was, these very cool achievements don't prevent you from calling all other people "idiotic", and from saying "they don't have a clue" without even looking at their designs, or even thinking that they might have had ideas different from yours.
Also, calling Git "patch-based" is quite wrong, but my point wasn't technical.
That isn't what I meant, I was actually talking about Perforce. RCS doesn't even handle multiple files.
I was about to reply point by point with arguments, but I changed my mind:
I don't think I'd be interested in that conversation, the comment above is worse than a public display of a very poor understanding of Git and Pijul: it also shows a complete ignorance of other actors in the market. It turns out the market leader for big repositories, Perforce, is itself based on RCS, possibly (but I don't know for sure, since Perforce is proprietary) because RCS scales much better to large binary assets than other solutions (I'd argue Pijul solves that, but that's beside my point).
I am a big fan of Git, Mercurial and Darcs myself, my co-author on Pijul was actually a maintainer of Darcs for many years, and I'm actively collaborating with the maintainers of Mercurial at the moment. And even though Perforce and Plastic are closed-source, they do solve one problem (scalability) which distributed systems are only beginning to understand (if there's one thing I think we've achieved with Pijul, it is about getting beyond the "distributed vs. scalable" trade-off).
Here's my take on the comment above: I don't think you can build good systems without understanding how others are made. There are no free lunches, no silver bullets, no geniuses. Only good ideas, bibliography, and hard work.
Thanks, I'm glad it was useful!
As an example, the type of channels/branches in Pijul contains things equivalent to `BTreeMap<String, (BTreeMap<Vertex, Vec<Edge>>, BTreeMap<u64, Change>)>`, and failing to update the pointers can result in corruption. Of course one can provide a safe interface on top of that (which I do in Pijul), but that depends on the types, so it isn't done yet (macros to derive safe implementations would be a really cool project btw).
Edit: to answer your question more specifically, small sectors might make it silently non-atomic, but there are workarounds (for example by using an extra level of indirection).
2. We often make the assumption that pages can be flushed to disk atomically, which is not very far from true on modern disks. Because the only update that really needs to be atomic is for these tables, they're all stored on one page. Since these pointers are all 64 bits in size (since 4k = 2^12, 12 bits in each of these are 0s), after a header of a few bytes, that leaves about 500 positions. The first few are probably more often atomic than the last ones, especially on very old disks.
3. It's still very important, but less so, because the Pijul repositories are much lighter on disk. Copying 1Mb is fast, but copying 100Mb is 100 times slower. I still like this feature, because (1) it costs almost nothing extra and (2) I really like the implementation.
Probably. The patch format is very unlikely to change. The repository format may change a little bit still.
I'd say it's probably ok to try and learn it now, but you should maybe wait for a few weeks before using it for something serious. On the other hand, we use it for itself, and I use it personally for most of my projects.
Applying a patch in Pijul is in O(p c log n), where p is the size of the patch and c the size of the largest "deletion-insertion conflict" p is involved in, where a "deletion-insertion conflict" is a situation where Alice deletes a block of text while Bob adds stuff in that same block.
Note that this is a rough bound, since all non-conflicting operations in a patch are in O(log n), except those involved in a "deletion-insertion conflict", which are in O(c log n).
So, Pijul is in fact faster than Git for merging (and rebasing). The only tradeoff at the moment is that going arbitrarily far back in history isn't as fast as it could be (this will be fixed very soon).
I don't expect the semantics to change much. The things that need polishing are probably on documentation, error messages, and network. The latest bugs in Pijul itself (i.e. not Libpijul) included handling the case where your HTTP connection drops while you're downloading, and keeping your connection busy, that sort of stuff.
- Alice splits a file into multiple smaller files. - In parallel, Bob changes one function.
At the moment, when they merge, they will get a conflict (both the same conflict, obviously), but I think it could be possible to remove the conflict.
There are two things I'm working on at the moment, and when these are done I believe 1.0 will be super close:
- Future-proofing the formats. There is one major thing I'd like to address before calling it 1.0, and that's code refactoring. It isn't supported at the moment, and Pijul is perfectly usable without that, but I'd like to keep the possibility of implementing it in the future open. I believe the current formats allow that, but I still need to be 100% certain.
- Merging two giant histories takes a time linear in the size of the history on the remote repository. Of course this isn't an operation you want to do very often, but it may become problematic for truly gigantic repositories, and I know how to fix that. This is planned to be a separate project at the moment, but it does require some support from the libpijul API.
Also, pijul log still depends on less, and there are some problems related to that. This is going to be fixed soon.
Something like that is announced at the end of the post ;-) It will hopefully be possible within a few days. The design is mostly done (at least for the basic feature), but my solution changes the implementation quite a bit, so I would expect temporary breakages (we're still in alpha, after all!).
This means for example that when you solve a conflict on a branch, and pull from the same remote, the conflict won't reappear (no `git rerere` needed).
Or that you can push a conflict resolution to a remote, even if that remote is in another state: as long as the remote has the conflicting patches, the resolution will work.
I explained it in a previous blog post: https://pijul.org/posts/2020-11-07-towards-1.0/#version-iden...
About recent features, I used to be very careful about using a stable Rust that was at least a few weeks old, or one generation behind. But the latest release got an unexpected number of contributors to the project, and we didn't even have a CI system!
We do have a basic CI now (https://nest.pijul.com/pijul/pijul/ci), so that's one thing we definitely want to check.
- Git does have spurious conflicts coming back for no reason. The problem is so real that there's even a `rerere` command to take care of it.
- Pijul has "minimal" conflicts, in the sense that these are conflicts you can't possibly make sense of without human guidance. You get the guarantee that fixed conflicts don't come back. Unfortunately, "obvious" cases are never completely obvious: let's say Alice and Bob both add a case to a C function, and both change `#define NUMBER_OF_CASES 42` to `#define NUMBER_OF_CASES 43`.
cargo install pijul --version 1.0.0-alpha.24
Difficulties to compile have been reported on Windows, we're still trying to figure out a Windows CI pipeline.
> But, this seems to require semantic level understanding of the code
No: you may very well have two different orders yielding the same output, even though the output is incorrect.
> This method seems to rely inherently on the fact that git commits are ordered. How would this work with a pijul-based flow?
Good question. Our CI over at https://nest.pijul.com/pijul/pijul/ci works in the same way. Patches are ordered in Pijul too, but the order is local. In your example, if Alice wins, Bob will pull her patch, fix his code, which will produce another patch. He will then send it to CI again (or directly to main if he's careless), and Alice will pull both Bob's original patch and the fix.
In the end, they will have different histories, but the exact same contents.
> But the ultimate goal is to produce a version of the file.
The fact that the end result is a version doesn't mean versions should be the main object. If you add a semicolon to a file, the actions you do are "press the ; key", not "create an entire version from scratch that has a semicolon there".
> If you want a diff management system, play around with quilt
You're doing it again! Why pick the least suitable tools for the jobs? Quilt is not a patch-based version control system, Pijul and Darcs are.