Jujutsu – A Git-compatible DVCS that is both simple and powerful
github.com
github.com
The Git compat seems like a great idea for this to really take off. My team is totally PR based though so if/when doing (Git compatible) feature branches lands in JJ I'm excited to switch.
I'm trying to find the part of the docs that refers to this functionality as missing but I can't -- does jj not have the ability to create, update, pull from, merge branches, etc?
Supported but not great I guess?
FYI, here's one way to use them with GitHub PRs:
1. Create a fork via GitHub's web UI 2. Clone using the SSH protocol (`jj git clone git@github.com:...`) 3. Make your changes and "commit" (`jj close/commit`) 4. Create a branch pointing to the commit you just created (`jj branch my-feature -r @-`) 5. Push it to your fork (`jj git push`)
So that doesn't seem too onerous, I guess? Maybe the "too many manual steps" are to clean up afterwards. I just tried it in a test repo and deleting the branch from the fork and then `jj git fetch` from there does delete the local branch, but it doesn't abandon the commits, so you need to run `jj abandon` on them if you don't want them anymore.
I honestly wish git forced `-p` for operations that support it. I’ve worked on too many teams where people would just commit everything in their working directory, and it would inevitably make reviewing their PRs a nightmare. Particularly with times where changes were accidentally committed wholesale that shouldn’t have been part of that set of changes.
JJ seems to workaround this in the other direction, by making the concept of commits and rebases much more lightweight, which i think is a refreshing enough take that I'd like to try it.
But that's what they've tested. I've had far more problems in the other direction, where the commit doesn't contain the complete set of things that it's supposed to but because all the tooling - every single IDE and compiler - is looking at the working directory not the index, I've missed something.
The index is definitely confusing for new users and users of other VCS.
> having a staging area where chunks are added piecemeal is an enormous benefit.
It would be quite fun if we could have hierarchical commits, so I could add bits to a commit without having to squash/amend. Then you'd see the top-level work item as a "commit" and the individual changes as "subcommits".
There should absolutely be better tooling around it, so that these piecemeal commits can be tested in isolation from one another. That’s a far better approach than just throwing up our hands and committing everything in the working tree, even if half of it has nothing to do with what’s intended.
Perhaps the focus should then be on making it easier to get that junk out of the working tree? Better stashing?
Would something like Mercurial's queues be something what you're looking for?
* https://stevelosh.com/blog/2010/08/a-git-users-guide-to-merc...
See especially "…with Two (or More) Patches" onwards.
https://pre-commit.com/ helps with that.
> It would be quite fun if we could have hierarchical commits, so I could add bits to a commit without having to squash/amend.
Yes! I've been saying this for years.
Branch merge commits are already this, kind of. However their UI sucks and there's an idea gap which prevents them from being fully understood as such "supercommits" composed of subcommits.
For one, the user should be forced to give meaningful, high level commit messages for merge commits and the tooling should fold commits made on a merge branch into the merge commit by default (effectively linearizing the merged history).
Ideally you'd have a data model that keeps a "what is this" description for branches that can be updated as long as the branch lives and that gets moved to the commit message on merge. Could this be done with some combination of convention and hooks? I guess a file in the root that is always kept on target/into branch state, with the lost file content of the source/from branch moved to the commit message would be all that's needed?
> Branch merge commits are already this, kind of. However their UI sucks and there's an idea gap which prevents them from being fully understood as such "supercommits" composed of subcommits.
Isn't that "idea gap" mainly the fault of all these weird newfangled git workflows that discourage or outright forbid branching? If you have no branches to merge, you can't have a merge commit to act as a "supercommit".
Call your "supercommit" a "feature", and what you need to implement it is... A feature branch.
And they always claim to review and clean their commits before pushing, or at least before merging, but never do.
See https://github.com/martinvonz/jj/blob/main/docs/branches.md for more information.
It's a bit strange to see "jj st" will automatically add all files to the "working copy" commit. This means that when I initially create my project, run "npm install" to install 500 MB of dependencies, and then run "jj st" to figure out what to include in my first commit, the command is going to copy all of that into a commit object for no reason. I don't think I like that behavior, to be honest. Is this a requirement of any of the other unique features? Could this be turned off, or modified to be an explicit operation?
[append]
To be clear, the fact that it's `jj st` is actually a big part of the problem for me: this is the command I use to figure out what to put into my `.gitignore` in the first place! I don't think that any of the "read-only" commands should be making these kind of changes.
Of course there are major cons.
It might not be technically the most superior way to do things, but given the widespread use, tooling around it need to be compatible.
This kind of thing is also pretty much standard for most programming languages today.
No, it is not. Have a look at java, golang or rust. None of those languages advocate for embedding deps in your source tree as it's quite clearly a terrible anti-pattern to follow.
I wouldn't consider how JavaScript does stuff a good example.
Perhaps that's a good time to use the undo operation.
That's really interesting. Having to manage stashes is annoying.
What’s also great is that you don’t need to do a pull in the other folder. Before I discovered worktree, I had my repository checked out in two places, so was either pulling both constantly, or when I needed the alternate, it was often very far behind and I had to pull a lot.
The great thing about worktrees is that everything local is still available in the worktree and vice-versa, you can even stash push on the one side and pop on the other. You also don’t need to push or pull on either side first.
I have noticed that for submodules, although they share the .git folder, they don’t share references, so you can’t see local branches and stashes between the worktrees, but sub module pulls are still quicker since they share objects so those don’t need re-pulled on the other folder.
When you're done working in a separately cloned repository in another folder, if you're anything like me, before deleting it (via `rm -rf`) you'll want to check very carefully for additional branches, unpushed work, anything stashed. Deleting an entire repository that's been around for a while is a risky operation in that it may have accumulated other unrelated work in addition to the current branch (which was the primary reason for the clone), and you'll want to check carefully before deleting the whole lot.
Additional worktrees within the same repository are great for medium-term ephemeral separate strands of work. They can be created and removed without concern.
Either it wouldn't have the right commit checked out anymore, or git would have to miraculously change what files are checked out in it -- which would likely come as a big surprise to whatever you were doing in that working tree.
Ergo, it is forbidden.
I periodically find myself creating a new (temporary) branch referring to the same commit when I want a new worktree looking at the same place, or just create a new worktree with a detached HEAD looking at the same commit.
What happens if you make a commit in dir1 and dir2 is on the same branch. Absolutely nothing, until you fetch, and worktrees could work the same way. If there is any ambiguity left you could prefix the branch names similar to the remotes origin/master, worktree2/master.
The only sane way to use worktrees today is, like you say, with detached heads. Which well…isn’t that sane for many other reasons.
I did not say that was the only sane way to use worktrees. I frequently use them for separate strands of ongoing work (on separate branches per work strand, named appropriately). Less common (for me) is the use case of easy reference to older versions, with more ephemeral worktrees checking out a tag, a new branch (created for the purpose, and short-lived), or a particular commit (viewed as a detached HEAD).
The way I use worktrees I'd say the majority are read-only anyhow, so abiding by restrictions because it might get confusing if a commit were made is pretty annoying.
The bigger picture is that the project is trying to provide a better interface on top of the existing git data store. As someone who used darcs for a major project before Git dominated DVCS, there are much better user interfaces than what Git offers, and attempts to improve upon Git is a diversity to encourage.
Unlike picking your own text editor, a whole team has to agree about the choice of DVCS, making it harder to try new ones.
The beauty of a solution with a Git-compatible data store is it allows some people on the team to experiment with it while still collaborating with people using Git.
When I'm adding a change, on the contrary, it's something I wrote minutes to seconds ago, I know what's in that code, and by adding that I'm kind of making a statement that this wasn't some random bullshit I do with the code, but something I intend to keep (even if later I'll completely change that while interactively rebasing or something like that).
On the other hand, I didn't complain that much back in the day when I was using SVN. But I feel like my current workflow that employs most unique features of git is actually much better. Basically, horribly inconsistent CLI is the only complaint I have about git.
This line of reasoning works really well for those who understand (and want to understand) how git works. I know plenty of people who don't care about their development tools (e.g. scientists), and for them the index is a chore in the way of more important problems.
Yeah, you should have an appropriately comprehensive .gitignore file, that's good practice in any case.
But I felt like this before when I switched from subversion to git (“why would I download all history, always???”) so, who knows.
And maybe I'm too brainwashed by Mercurial, but to me the index is nothing but a single weird commit with only downsides (why do we need a UI to work with the index that's different from the one to interact with commits, although the capabilities should be the same? Why being limited to a single index and not several? Why the different versioning/shareability characteristics, etc).
I largely prefer Mercurial's "public vs draft" strategy and the "commit what you have and we give you the tools to tidy up the series whenever you feel like it". In practice it means that you have as many "indexes" as your series is long, and with hg's mutable-history and amazing history-rewriting extensions like absorb, it's much more convenient, fast and safe to work with than the git inconsistent equivalents.
My workflow is basically just change whatever I want to get to my goal. When I finally get the behavior I want, stage the changes and stash the rest. Once I've verified that everything works, commit the stage and drop the stash. I don't have to think about anything not directly related to my commit. Feels like with tools like jujutsu, pijul, mercurial...I'd have a lot of manual cleanup to do _before_ committing...and then what if I accidentally clean up something I didn't realize I needed? It's gone. Whereas, with git I can just pop the stash and stage whatever piece I missed.
2) Are there any potential issues one has to be aware of when using jj contributing to git repository?
I remember when I was the only team member using git-svn plugin my coworkers were confused when I svn-committed many commits at once with some of them breaking the time order of the svn history (as git-svn commits in svn were recorded with git timestamp and not the actual svn- commit timestamps)
Very few. The disadvantages are generally much larger. The main advantage is that you won't run into a (harmless) race that happens once in a while with the git backend (https://github.com/martinvonz/jj/issues/27). Disadvantages include not being able to interact with git repo and performance problems (both size and speed).
I should add a note about this to the README.
The backend exists mostly to prove that it's possible and to make sure that the backend API doesn't become tied to Git.
> 2) Are there any potential issues one has to be aware of when using jj contributing to git repository?
The main one is that you should avoid pushing commits with conflicts. It won't corrupt the remote repo, but git clients won't know what to do with the conflicted files. The fix is to fix the conflict and force push the conflict-free commit.
I'll file a bug to prevent pushing commits with conflicts.
This seems kind of similar to pushing git files with conflicts with the git client? Maybe the Jujutsu formatting is slightly different, but the concept is the same.
Do you expect this to always be the case or are you waiting for inspiration/collaboration to make a better go of it?
git got the data model really right, and the user space really wrong. The data model is more important than the user space.
This is the first project I've seen which appears to understand and respect /how/ and /why/ the git data model is so elegant.
With a version-based model, how you compare them is determined by the user space, whether that's merges, or viewing history. Even compression isn't all that fundamental to much of anything; it's possible to implement underneath the system, eliminating whatever redundancy you want. With patch-based systems, all of that comes baked into the underlying representation.
On a deeper level, this feeds into how git can be distributed, robust, yet very simple.
It'd be possible to write a really good user space around the git data model if someone had the time and inclination.
My personal gripes:
* I like to commit every time I change a few lines of code. To have a clean version history, git requires me to rebase all of those commits. I'd like to have "major" and "minor" commits, which the git data model can support in several different ways (as a list, a tree, or DAG; I prefer the tree model). The user space makes anything like this incredibly painful. It encourages rewriting history, which is actually rather bad.
* Large files are managed very, very badly. It's ironic since the git data model is ideal for things with large files.
* Related, partial trees aren't really supported. This gives scaling issues. In many cases, I just want to work with a few files, or just the latest commit. Some of this is technically possible but practically painful enough to be not worthwhile.
* Nested projects / submodules.
Of those, only submodules might require changes to the underlying data model.
For those interested in the different perspectives on how to weigh those priorities, I recommend starting here:
The git comparison docs have:
> Start working on a new change based on the <main> branch -- jj co main
Did you consider the recent git nomenclature change to use "switch" for branch operations, and "co" for file operations? I actually can't tell from so deep in my "git Stockholm syndrome" whether that distinction is really hard on new users or not, but the fact that git expended the energy meant they thought it was meaningful
And, do you have plans on supporting signed commits, either via gpg or the newfound SSH key signing?
I'm super excited to try out the conflict resolution mechanism, because that's a major painpoint for my long-lived PR branches
> Did you consider the recent git nomenclature change to use "switch" for branch operations, and "co" for file operations?
Actually, isn't "restore" for file operations? My impression was that everyone agrees that `git checkout` does too many different things. In particular, it's both for switching branches and for restoring file content. So they added the new `git switch` and `git restore` with limited scope. I strongly suspect that they would have used `git checkout` for the former if that wasn't already taken by the existing command.
> And, do you have plans on supporting signed commits, either via gpg or the newfound SSH key signing?
No, that's not something I've even started thinking about. I'll have to read up on how it works in Git first. Patches welcome, though :)
> I'm super excited to try out the conflict resolution mechanism, because that's a major painpoint for my long-lived PR branches
You mean so you can continuously rebase them without having to resolve conflicts right away? Yes, that's one of the benefits of first-class conflicts. Another benefit, which took me a long time to realize how useful it is, is the auto-rebase feature.
I opened an issue and provided links to the docs, and a link to the current way that git implements it: https://github.com/martinvonz/jj/issues/58
As I was trying to find the correct command in jj for where to add the new argument, I realized that this may not play nice-nice with jj's commit-everything model, so I'm actually prepared for the issue to be closed WONTFIX :-)
As for the patches welcome part, I notice that the Google CLA bot is on your repo. While I do have the CLA signed for my Gmail address, it seemed like some major tomfoolery to try and add my current github email address (and the one which backs my GPG key, to bring this full circle!) to the existing CLA process. Do you intend to keep that CLA mechanism in place, or was it just an oversight?
Yes, I suppose you might not want to sign every working copy commit, but as you noted on the issue, it would probably make sense on `jj close/commit` (aliases).
> Do you intend to keep that CLA mechanism in place
I started working on this project internally and then open-sourced it. I don't think I'm allowed to remove the CLA bot. I understand that it's annoying :(
(I post this somewhat selfishly, as I'm basically describing how I wish git worked ;)
That said though, you are probably right something will eventually supplant git. It would be arrogant to think that my failure to imagine something better means there isn't anything.
For what it's worth, SVN was pretty straightforward and worked well enough at the time. Later, Mercurial addressed SVN's deficiencies with a familiar interface.
I think the way Git handles renames is ugly – it doesn't actually record them, it just tries to guess when they occur based on file contents (inevitably imperfect). I think Subversion handled this better.
The problem is that a Git tree object only contains file name, file mode, and blob/subtree hash. If it also contained some kind of "file ID" (such as a UUID), then you could track renames properly – renaming a file would change its name but not its file ID, so you'd track the rename properly, even if the contents changed at the same time.
Given they didn't do that... maybe create a file-id Git attribute? (Alas, "git mv" doesn't update .gitattributes, so renaming a file forgets the attributes; but it could learn.)
I'm not quite sure why Linus decided not to make renames/moves/copies explicit/strict and my only guess is it simplified things. Being able to say with 100% certainty all the time that something was renamed or copied is extremely handy, but with Git we just get a percentage like the following shows:
https://oss.gitsense.com/insights/github?bq=move%2Bpull-age%...
I guess knowing the mapper.go file in the example above was likely copied is good to know, but it would be much better to know this with 100% certainty.
Full disclosure: The link above goes to my tool
I don't think it has anything to do with the fact that Git is distributed. It is a question of how rich the repository data model is. The richness of the repository data model is an orthogonal concern from distributed-vs-centralised.
I think Linus just wanted to keep it as simple as possible – but, maybe in some areas he made it too simple. The Subversion developers on the contrary, maybe went too far in the other direction.
I think it does since meta-data increases merge complexity, which is probably a headache when dealing with a distributed system. With a centralized system, you are basically creating a lock when you do a rename/copy/etc. which doesn't translate well to distributed systems.
With Git as it is currently implemented, you only need to worry about one state which is the tree. With meta-data, the number of states that needs to be tracked can increase significantly.
Having said all of that, I guess it probably wouldn't be too hard to implement a layer on top of Git that ensures every directory has something like a .git.meta file and have hooks that ensures all renames,copies,etc. are recorded in the meta-data file.
I disagree. In Git, merging always happens locally, between two local commits; there is nothing distributed about merging itself.
> With a centralized system, you are basically creating a lock when you do a rename/copy/etc. which doesn't translate well to distributed systems.
I don't think that is really true. If I rename a file in Subversion, that doesn't involve "creating a lock". I rename the file locally (using "svn mv"), Subversion locally remembers I have done it, but doesn't send anything to the server. When I want to commit, it then contacts the server and uploads the commit data, including the rename tracking info. Now, that commit operation takes a write lock on the branch – to prevent any other commit for that branch being processed at the same time – but it is the exact same write lock regardless of whether renames are involved or not. If we take Subversion as an example of a centralised version control system with rename tracking, the rename tracking and the locking are orthogonal. (When I say "locking" here, I mean the internal locks within the Subversion server, not the end-user-visible advisory locks you get with the "svn lock" command–that advisory locking feature is unrelated to the topic of rename tracking.)
Like Git, merging is essentially a local process in Subversion – the client constructs a merge commit, and then sends the completed merge commit to the server. The actual merge, meaning construction of the merge commit contents, always happens on the client side, at least as far as the core Subversion protocol is concerned.
In fact on the whole repo, since branches are just subdirectories. (Shows how long it has been since I've actually used Subversion, I'm starting to forget it.)
i fear to add such a feature additional information would need to be stored in the git repo itself which would require a change to git
A commit having multiple parents (a merge commit) maintains the order of those parents within the commit. The branch names are famously lost, but by convention you can say that the first parent should always be the main branch.
As with all other git things, the ux for this feature isn’t the best and it’s use varies wildly across tools and organizations.
To view the main branch log only you have to pass arcane -—left-only flags into git log.
Some people don’t know the difference of merging from branch to main vs the other way around, end result looks the same right. One such mistake messes up the history.
In fact, it's a pretty common branch-based flow. Our team pushes feature branches to remote, where they are vetted by QA, and we have a chance for review.
When pulling changes into any branch, we always `git pull --rebase`. When merging feature branches, we always `--no-ff`.
For small, local merges, we don't use `--no-ff` because it's useless noise. But if someone forgets out of habit... Oh well. I spend on average 0.000000001% of my time going through git log. And when I do, I tell it to show as a tree
Yeah, it's just a coincidence that the world Jujutsu means magic in japanese (呪術)
Removing the "convince my team to move away from git" part of a better DVCS could make jujutsu very popular.
Or other git compatible spinoffs could be made that follow this approach and explore other design spaces.
Right now
error[E0554]: `#![feature]` may not be used on the stable release channel
--> lib/src/lib.rs:15:12
|Out of curiosity: did you add support for multiple backends because you don’t think Git will ultimately be sufficient for the experience you want to create? Or did you just want to ensure that the tool didn’t become unnecessarily coupled to Git?
Awesome, that would enable a realistic no-risk path to adoption for most projects:
- trying jujutsu on work codebase
- if useful, tell a coworker and maybe they try it
- present internally about pros
- team tries and agrees its better, adopts it
- after some months confidence is built, give a presentation on features you could get with jujutsu native backend, gauge interests\worth
The basis for it is the read-tree command.
In git, every commit is a snapshot and not a delta. (Repeat that three times.)
Any arbitrary snapshots of files can be arranged into a git history. They don't even have to be related. We could take a tarball of GCC, and make that the first commit. Then a tarball of Clang and make that the second commit.
The read-tree command will read any commit's state into the index, and from there you can make a commit out of it.
To reorder some N commits you're looking at, save all of their hashes somewhere, and then rewind the branch: git reset --hard HEAD~N. Then use git read-tree to read those hashes in whatever order you want, committing them one by one. To reuse their commit messages you have to use commit -C <hash>, though most them likely don't make any sense, because the text of commit messages usually talks about changes between a commit and its main parent.
What will happen is that your history now as N commits which represent the same states as the N you had there before, in a different order. For instance if you reverse them, then the oldest commit has all the features (is the same code baseline) as what the newest HEAD was previously. And then the subsequent child commits basically remove all the changes that were made, so the latest now looks like what the N-th looked like.
How I sometimes use this:
Suppose I made a fairly complex change that I would like to break up so that it is presented as a sequence of two or more states of the code.
There are cases when the following workflow is easy: first I make a commit out of the changes. One commit with everything. That commit is what I want the final state to look like. Then I revert some of the changes to produce the state before that state. If I have mostly been adding things, this might consist of just deleting them. Or I can do a git reset --patch HEAD^ to selectively revert. I commit this penultimate state. Then repeat the process: revert some more changes from that one, commit and so on, as many times as I see fit.
So what I end up with is the states of the code I want to present as a history, but exactly in the wrong order. Using "git rebase -i" doesn't work nicely; there are ugly conflicts, and some of them have to be encountered twice. Here is where the git read-tree trick comes into play: I simply reverse those exact states of the code to put them in the right order on the branch. After that I might do a rebase -i just to reword the commit messages to frame the code states from the POV of being changes from their parents.
You might think: why not just "git commit --patch" from the original working state to produce commits in order. The reason is that doesn't always make sense. For that you had to remember to make the commit as you were working. Because say you refactored something and then made a change. You can't easily do a "git commit --patch" which separates the refactoring from the change. Sure, if you do the refactoring and then a commit, and then make the change, you are good. But suppose you've conflated them already; now what? You can commit everything and then back out the changes that were done on top of the refactoring, and commit that. Then reorder the two states.
I know this, but also I cannot reconcile this whit what happens when you cherry-pick a commit from another branch. I'm confused because cherry-pick really seems to be taking out the delta not the snapshot.
I'm thinking that cherry-pick takes that commit and makes a diff with its parent and then that's what you get when you call it. Is it how it works behind the scenes?
I'm also thinking that the fact that each commit has at least one parent means that, conceptually, we can use it as if it was a delta (at least in the case of commit with one parent only), if you get what I mean.
EDIT: I'm not familiar with the internals of Git, just a user. Commenting out of curiosity.
A cherry-pick deos not just do a diff with its parent 0 and then patch it onto your work. In many cases, that wouldn't work because that's not a three-way-merge, but a two-way merge, which has disadvantages.
However: there may be situations when the commits are so distant, it might make sense just to turn that one into a patch and work the patch onto your baseline. Even if that needs manual work, it can be simpler. You will only get conflicts (patch hunk rejects in that case) that are relevant to that change. I've had lots of experience working with patch stacks (both before and after Quilt was introduced to make that a bit easier) so I'm comfortable migrating patches from one code base to another without a three-way-diff process within version control.
Thank you for explaining the three-way merge between baseline, commit and common ancestor, makes sense.
The first hint that it's doing something interesting is that the implementation of cherry-pick is in revert.c, because it's implemented as a variant of revert: https://github.com/git/git/blob/v2.35.1/builtin/revert.c#L23...
Both the "revert" and "cherry-pick" operations initialize a sequencer object with a single operation in the sequence. (This is the same mechanism underlying "git rebase -i").
From there we can find the call to sequencer_pick_revisions, which leads us to the sequencer implementation, where there's a fast-path for ordinary cherry-picks leading us to a short function single_pick, which in turn calls do_pick_commit: https://github.com/git/git/blob/v2.35.1/sequencer.c#L2066
This operation then does the following:
- Determine what we're applying a diff on top of: https://github.com/git/git/blob/v2.35.1/sequencer.c#L2083-L2...
- Find a parent commit. If the commit has more than one parent, it requires you to specify a "-m" option indicating which parent to diff against. https://github.com/git/git/blob/v2.35.1/sequencer.c#L2109-L2...
- Set up the commit message and some other stuff.
- Set the variable "base" to the parent it found and "next" to the commit being cherry-picked. https://github.com/git/git/blob/v2.35.1/sequencer.c#L2187-L2...
- Assuming no merge strategy (or either of the standard strategies "ort" or "recursive") was specified on the command line, call do_recursive_merge. https://github.com/git/git/blob/v2.35.1/sequencer.c#L2237-L2...
In turn, do_recursive_merge calculates a head_tree from the current HEAD (https://github.com/git/git/blob/v2.35.1/sequencer.c#L645), and then calls either merge_incore_nonrecursive (the current default, from the new "ort" merge strategy) or merge_trees (from the older "recursive" merge strategy) with three trees (i.e., with no further ancestry information): the base is the parent it found, and the two sides being merged are your current HEAD and the (tree of) the commit being cherry-picked.
That is to say, at no point does it care whether the two commits even have a common ancestor! It's just doing an operation on trees. It is doing a three-way merge, yes, but the graph of the merge it's doing is one that potentially doesn't actually exist in reality.
Or, in other words, it's trying to compute the tree that could be equally well described as the result of
- applying the diff of the commit you're cherry-picking to your current HEAD
- applying the diff between your current HEAD and the parent of the commit you're cherry-picking to the commit you're cherry-picking
So this is more powerful than purely applying a diff with no information about the base of the diff, but it is very much like applying a diff.
One way you can test this without drilling into source code is to make two independent commit histories (using either two git repos, or git checkout --orphan, or whatever) where a commit in one history has a diff that would apply to a file in the other history if applied with the "patch" command. Then try cherry-picking it into the second history. It should work.
(In the revert case, it more or less does the same thing, just with "base" and "next" flipped. That is, if you have commits A, B, and C, and you want to revert B, it does a three-way merge where the base is the tree after B, one side is the tree after A, and the other side is the tree after C!)
$ diff3 -m my-file pick-file-parent pick-file> "it applies a diff between a specified commit and its parent on top of your current work"
this is in line with my intuitive understanding which was based purely on usage.
FYI, this is also how `jj undo` works, except that it's a three-way merge at the repo level (https://github.com/martinvonz/jj/blob/d9b364442e2246a734d600...). So that applies changes to branches, checkouts (think: git HEAD), and sets of anonymous heads. This is how you can undo an operation even if it wasn't the most recent one (just like you can `git revert` a commit that wasn't the most recent one).
A
B
state-of-B
state-of-A
pick A
x git revert HEAD
pick A
pick B
pick a
squash d
pick b
squash a
It's an abstraction inversion to be doing that. The system doesn't work with diffs and merges: that's just layering on top. The system contains snapshots, and so if you want to rearrange snapshots in a different order, that's the best abstraction layer to work at.
Anyways:
You start with:
original-state
A
final-state
B
intermediate-state
original-state
X
intermediate-state
Y
final-state
git revert HEAD
original-state
A
final-state
B
intermediate-state
revert (C)
final-state
pick A
squash B
reword C
Edit: but yes, you can achieve the same outcome with read-tree or checkout.
Edit2: come to think or it, you can probably do what you want with a couple git replace --graft and a noop filter-branch.
> In git, every commit is a snapshot and not a delta. (Repeat that three times.)
Yes, but every commit also is notionally a delta. That the delta is reconstructed as needed is not terribly important. It's best to accept a commit as both, a reference to a state of the world, and as a delta to the recorded parent(s).
Lastly, I avoid `git reset --hard`. I often have extant changes that I don't want to lose. Instead what I do is `git rebase -i --autostash`.
You lost me at "free from the index". The index is one of the most important parts of Git that makes my life easier. Opinionated DVCS UIs make my life harder -- all of them.
> The working copy is automatically committed
Right, so, the reason the index is powerful is that I get to do `git add -e`, `git commit`, and repeat until I'm done or ready to throw remaining changes away. I very much want the index / workspace distinction.
> Automatic rebase
> Comprehensive support for rewriting history
This is very welcome. At the very least this confirms something important: the fallacious argument that "rebase rewrites history, so it's eeeevil" is finally dead.
You can do that with git commit -p.
Most of which you can get on git show at least if they're relevant to “what have I added”. And of course you can also use git diff on commits if you need something super specific.
> For another, it has no side effects, unlike `git commit --amend -p`.
… “git commit --amend -p” is the replacement for subsequent “git add -p”, as the GP was talking about a workfliw where they’d intersperse staging stuff and looking at what they’d staged.
`jj split -r <commit>`, where `<commit>` may be the working copy commit (which is the default). See https://github.com/martinvonz/jj/blob/main/docs/git-comparis... for more examples.
: ; $EDITOR some-file ...
: ; $build
: ; git add -e # take a few chunks, maybe change them
: ; git diff --staged; git status -uno
: ; # lather, rinse, repeat
: ; git commit -m '...' -ev
: ;
: ; git status -uno; git diff
: ; git add -e # ..
: ; # lather, rinse, repeat until happy
: ;
: ; git fetch origin
: ; git rebase origin/master
: ; # fix any conflicts
: ;
: ; # continue until all bug fix / feature
: ; # activity for this issue is complete
: ;
: ; # review my changes:
: ; git log --patch origin/master..
: ;
: ; # if I have to clean my history a bit:
: ; git rebase -i origin/master
: ; # re-order commits, `edit` commits, `drop`
: ; # commits as needed
: ; $build
: ; # fix remaining issues...
: ;
: ; # finally ready to push:
: ; git push myclone HEAD:refs/heads/this-thing
: ;
: ; # open a PR
> detached HEAD mode
Ah, I see, not anymore...
If you click the link that text points you to (i.e. https://github.com/martinvonz/jj/blob/main/docs/git-comparis...), there's an explanation there for how to achieve the same workflows. I get that it's different, but I don't think it's worse. I consider myself a (former) git power user (I think I have ~90 patches in Git itself) and I've never missed the index since I switched to Mercurial ~7 years ago.
> This is very welcome.
Thanks :)
This is more confusing? Often times when debugging/writing a fix I would have extraneous code that I wouldn't want to commit. With an index I'm always sure if what I commit, but with this workflow you have to keep track of such stuff all the time and if you forget that stuff makes it in?
Not to mention that another benefit of an index is being able to change commits and git replaying your working diff.
i often have temporary files that i do not want to commit, nor do i want to add them to .gitignore (because i want to commit them later)
but then, i'll have to spend some time using jj split. if it is powerful enough then maybe the end result is just that those files only live in the last commit.
also, what happens on push? i'd never ever want the working copy to be pushed to the remote repo. i could not find anything in the documentation about that.
(according to the answer here: https://news.ycombinator.com/item?id=30399554 the working copy is not supposed to be pushed)
Sign me up!
Happy user for at least 6 years.
The sanity of Mercurial's UX cannot be overstated.
Also, I often rebase and `edit` commits to split them or undo parts of them.
Rebase and all this is all about making commits that have just the right content, and keeping history clean and linear. The tools have to make this possible and easy.
I get that git feels... barebones for this. You really have to understand what you're doing when using git like I do, so I get that it's not very accessible.
Better UIs are great, but on the other hand, we need to be able to get down to the low level.
IMO.
Interesting. I don't think I've heard anyone use `git add -e` before. It should be easy to add that feature, but it would be very low priority since so few users seem to like to manually edit patches.
> Also, I often rebase and `edit` commits to split them or undo parts of them.
You can do that by checking out the commit, then `jj squash/amend` (aliases) and all descendants will be automatically rebased on top, and branches pointing to them will be updated too. There's also `jj edit` for editing the changes in a commit without updating to it. And there's `jj split` for splitting a commit (without needing to update to it).
> Rebase and all this is all about making commits that have just the right content, and keeping history clean and linear. The tools have to make this possible and easy.
Yes, that's definitely a goal. I think Jujutsu does a much better job at that than Git does. Did you read https://github.com/martinvonz/jj#comprehensive-support-for-r...?
But even though I might use a tool specifically designed for user-friendly commit splitting, I still want: `git add -e`, `git diff --staged` (to see what I added with `git add -e`) vs `git diff` (to see what I left out), and `git commit` w/o `-a` to commit the contents of the index. This is easier for me than `$WHATEVER commit` followed by `$WHATEVER`'s commit splitting method.
That said, I have to congratulate you again on not taking the ugh-rebase-rewrites-history/history-rewrite-bad road. This is a huge step forward for DVCS.
It may well be that what rebase workflows needed was a UI that non-power users could use.
While we're at it, I wonder what you think of this: https://gist.github.com/nicowilliams/ea2fa2b445c2db50d2ee650...
EDIT: The idea of being able to suspend/resume and push/fetch in-progress merges/rebases is very cool indeed. It's hard to tell if it could speed up the task of rebasing across thousands of commits or not -- the README doesn't say. I like Viktor's script because it's very focused on one thing: quickly rebasing local commits across thousands of new upstream commits with a minimum of conflicts -- a simple solution to a simple problem statement (though the problem can feel huge when you have it).
From the README, it looks like there's robust support for editing and splitting commits. So maybe in practice the flow is similar to using the index, with the added the added benefit that your work is backed via commits along the way, and the simplicity of not having the index as an extra concept.
In general when exvaluating X from the perspective of Y, we will immediately see the thing's about Y we like that X lacks; it takes more time to see if perhaps in the fuller context of X those things are not necessary.
You don’t need the index for that. In fact I’d say it gets in the way because the presence of the one means less pressure on improving the ability to edit commits: while it’s easy to add stuff to HEAD it’s much more painful to remove content from it.
If that is solved, then the value of the index drops precipitously, because you can create a commit with a purpose and select its content instead of having to do it the other way around then forgetting what change you were trying to craft.
Mercurial has an 'index' / staging area, but not exposed by default. You can access it with some extra CLI options, but there is an optional idea that may be 'better' and worth looking into:
> If you need the index, you can gain its behavior (with many additional options) with mercurial queues [1] (MQ).[2] Simple addition of changes to the index can be imitated by just building up a commit with hg commit --amend (optionally with --secret, see phases [3]).
* https://www.mercurial-scm.org/wiki/GitConcepts#Git.27s_stagi...
[1] is "A Git User's Guide to Mercurial Queues"
* https://stevelosh.com/blog/2010/08/a-git-users-guide-to-merc...
MQs (optionally) expand on the idea of only a single staging area:
> This single "intermediate" area is where git stops. For many workflows it's enough, but if you want more power MQ has you covered.
> MQ is called Mercurial Queues for a reason. You can have more than one patch in your queue, which means you can have multiple "intermediate" areas if you need them.
If you only want to use one queue/index then that's fine too.
That Steve Losh post is from 2010 and it was mainly highlighting a workflow that was popular at the time for a particular use case. It also highlighted how Mercurial’s plug-in architecture can be used to to support different workflows.
Fast forward to the present and the use of MQ isn’t really a thing anymore, but is available for those who want it.
Check out https://octobus.net/blog/2020-11-26-modern-mercurial.html for modern Mercurial practices.
Furthermore, the index is effectively a pseudo commit without a commit message. Any workflow using the index can be implemented in terms of actual commits itself.
I think because Git doesn't have strong usability in general and especially around history rewriting, many Git users feel that the index or an index equivalent is somehow a required feature of a VCS because Git's shortcomings give that illusion. However, if you use a VCS with better history rewriting (such as Mercurial with evolve), you'll likely come around to my opinion that the index can be jettisoned without meaningful loss of functionality or productivity.
Perhaps there's room for new UIs.
All I'm saying is I need this power. And it has to be easy to reach for it.
That said, building a version control system seems like a problem similar to a social network: the network affect causes an enormous amount of friction. i.e. most people don't want to use it until other people are using it. A vicious cycle.
The fact that it's compatible with git as a backend is really great though, and is probably the key to driving adoption. However I have several immediate thoughts that come to mind that cause me hesitancy. Answers in the README.md would be super helpful for marketing to me (and deep apology if it's there and I missed it):
1. How does it look to others working on the repo using just git? For example, "when the working copy is automatically committed," where does it go? is there a remote branch that someone could see if they `git fetch` while I'm working? I often put all sorts of things in the code that I don't want to have committed, ranging from harmless (a bunch of personal comments or `IO.puts()`), to very important (API keys and such that I'm testing with. I always move them to env vars before committing but for first-pass testing to prove the concept I "hardcode" them at first).[1]
2. Similar to "how does it look to others," what sort of burden does "all operations you perform in the repo are recorded, along with a snapshot of the repo state after the operation" put on the git backend? If I'm hacking on an ffmpeg script and I (temporarily) copy a 4GB mp4 file into the working directory so I can easily run `./mycode video.mp4`, does that whole thing get committed, potentially every time it changes? That could easily turn into a 40GB set of changes.[1]
3. Do you have to use the jj backend to get all the feature? For example, with a git backend could you get the two things mentioned above and also "Conflicts can be recorded in commits" as well?
A quick section in the README.md about which features require the jj backend instead of git would be super helpful, and if written well could answer all of the questions above.
To sum up my comment in a TL;DR: To really sell me on a project like this it has to work with git, and being able to quickly tell which features I can use with git, would make this substantially more interesting to me.
[1]: Related: https://xkcd.com/1172/
It becomes a regular git commit with a git ref called something like `refs/jj/keep/9f1a0fb0-a0aa-4a4b-922f-d6d48687996a` pointing to it (to prevent GC). It won't get fetched or pushed by default, except with `--mirror`, I think.
> If I'm hacking on an ffmpeg script and I (temporarily) copy a 4GB mp4 file into the working directory so I can easily run `./mycode video.mp4`
Yes! You'll have to more diligent about keeping your .gitignore (or .git/info/exclude, etc) file updated. I plan to add commands for fixing such mistakes by forgetting the commit.
> Do you have to use the jj backend to get all the feature?
Nope, conflicts work with the git backend as well. https://github.com/martinvonz/jj/blob/main/docs/git-compatib... has some info.
> To sum up my comment in a TL;DR: To really sell me on a project like this it has to work with git, and being able to quickly tell which features I can use with git, would make this substantially more interesting to me.
Makes sense. Thanks for the suggestion! I'll put it on my TODO list.
What, it isn't actually a decentralised distributed system any more?!?
> I'd love to just dump a bare repo to a syncthing share and collaborate with a small group of people accessing it.
Are you saying one can't do that with the latest versions of git?
WTF, when did this happen and how can I have missed what must have been huge news when it happened?
You can have everyone manually set up a git hosting env on their dev machine, then have everyone manually add a remote for every other developer's box, but it sure isn't convenient.
And if you settle on sharing one central box as the canonical node (which IMO is the right answer), you're no longer truly using a distributed system - it's a de facto centralized system.
Ah, OK, gotcha. Thanks!
> You can have everyone manually set up a git hosting env on their dev machine, then have everyone manually add a remote for every other developer's box, but it sure isn't convenient.
Yeah. But fortunately not all that much of a hassle for a small(ish) dev team.
> And if you settle on sharing one central box as the canonical node (which IMO is the right answer),
Yeah, we called it "the Dev server". :-)
> you're no longer truly using a distributed system - it's a de facto centralized system.
But in a corporate (or even Open Source project?) environment, The Powers That Be probably want that kind of / that much centralisation anyway.
Agreed. I have used git in genuinely decentralized ways and have concluded that in practice centralization is the right answer for almost all projects.
Even though having the full history locally is a transformative improvement pet Subversion's model, in practice it's really helpful to know there's a canonical source of truth and to coordinate around it.
Yeah, I've probably just simply been lucky so far: IME with git, which hasn't been very extensive -- and only with a quite small dev team -- that just never happened to occur.
I suppose I can still tell you that the features were not inspired by Piper without revealing whether Piper has them :)
I wonder how this works out for large files and cleaning them up.
I can’t recall the last time a coworker who liked things because they were powerful didn’t end up being untrustworthy. Even dangerous. It’s like nobody remembers the Principle of Least Power.
That said, I will take someone obsessed with “powerful” over “flexible” any day of the week.
The Leatherman is part of my EDC, along with an LED flashlight with some respectable lumens so I don't have to use my phone to see in the dark.
In software this is known as the Unix Philosophy, but we violate it quite often, and call those tools 'powerful' or 'flexible'. Everything is a Swiss Army Knife all the time, and we aren't self-aware enough to see how consistently - and sometimes badly - we struggle with this.
But you can't tell an addict they're an addict. They will fight you to the death about how they Don't Have a Problem.
By allowing a Git backend to use the Jujutsu frontend it could develop a following.
This is the biggest thing keeping me using mercurial, the fact that it has subrepos that work.
Try mercurial to see how to do it correctly.
On the positive side: It signifies a certain likelihood of memory and data safety that you won’t get in most other languages.
Jean Baudrillard would call it another level of simulation.
Of course I cannot remember how exactly the "git rebase" command works and have to look it up every time, same as all those switches to the "simpler" commands.
I guess it all depends on what you need. Python looks really complicated too, if you start by browsing the function and class reference. But for someone starting to learn programming, it is apparently pretty easy (or so I've heard)
We all know the dig here - Git is not simple.
Like many tools, Git has evolved significantly over the years. Git today was not like Git 10 years ago.
Also, like many replacements to existing tools and software, they always start out simple and beautiful. Then they grow in complexity to serve the domain. The reason Git is complicated - not "simple" - is mostly because version control _is_ complex.
I also don't agree that Git is hard to use. I feel it is an odd goal to try to make everything - even tools that experts use - simple to use, when they are fundamentally not simple. I feel like Git is sufficiently complex - no more than it needs to be and certainly not less.
Perhaps the biggest mistake (IMO) was to expose the index to the user. I happened to just watch https://www.youtube.com/watch?v=31XZYMjg93o (by the person behind Gitless). They explain the issues well there.
It really feels to me like a commit identifier should be the combination of a monotonically increasing number and a hash of the content, rather than an either-or. If I merge or rebase branches in git, I lose the index, just as I would in subversion. But at least in svn I have some notion of how far back commit 12345 is in the tree. ac4def2 could be HEAD or four years ago.
Shots fired.
After using git for 6 years git still terrifies me. After 9 months of svn I performed open heart surgery to remove a 1GB zip file that some dummy merged before anyone thought to stop him. It was at least 18 months before I tried something similar with git and I made copies of my copies to make sure I didn't fuck it up. And I still almost needed a do-over.
The level of dread I have using git is pretty close to how I feel when using a newly sharpened chef's knife - hypervigilance. And that feeling gets stronger when I hand that knife to someone else. Be very careful, that's super sharp... hold still, I'll get you a bandaid. Now you get a mini lecture on how to hold a knife without cutting yourself next time.
It's not hard to be better at git than 85% of people, most devs I've met don't understand the basics beyond pull commit push.
But it’s certainly not the easiest thing to mess up a Git repository to the point of losing data; the reflog keeps it all intact for at least a month unless you very deliberately tell it not to (see the “CHECKLIST FOR SHRINKING A REPOSITORY” section of git-filter-branch(1) for sample invocations after such an edit), and I’d be much more comfortable about open heart surgery on Git’s model than on SVN’s.
First git project we had a guy who was bad at merges, and his solution was to avoid them as long as he could, which just made things worse. He also disliked another engineer. That engineer was definitely making mistakes but at the end of the day his Dunning Krueger was causing me less trouble than merge boy’s. Also suspect a little ageism, but would not have been able to defend that suspicion.
So one day merge boy was loudly exclaiming that old guy had written a terrible bug and the hit history proved it. Only the bug in question was something I specifically looked for in the code review, and had been happy to find that he had gotten it right. But here in git annotate it shows he got it wrong. What the hell?
So I step through the commit history one commit at a time, sure enough the code was initially correct… until merge boy did a merge and fucked up the three way. Again. And got old guy’s name to show up in annotate. I didn’t even know you could do that.