FreeBSD src repository is transitioning from Subversion to Git this weekend | Hacker News Reader

FreeBSD src repository is transitioning from Subversion to Git this weekend

lists.freebsd.org

212 pointscnst74 comments

Almost hard to believe it's finally happening! Couldn't we have gone with Git all along, without Subversion as an intermediary step for about a decade?

Anyhow, per https://wiki.freebsd.org/git, looks like GitHub/GitLab will have the new stuff on Tuesday, 2020-12-22T10:00Z; this'll likely require manual actions if you're exporting Git from GitHub. Personally, I'll probably be switching over to exporting directly from FreeBSD.org at that time, because GitHub still doesn't even support IPv6.

My source is Wikipedia, but FreeBSD is so much older than git. Git is only a decade and a half old (from 2005).

https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committ...

freebsd switched to subversion in 2008.

I would guess that git was just too new in 2008.

It wasn't just new, CVS to subversion was nearly drop-in change. It didn't require change in process - FreeBSD project could have kept its Cathedral style process.

Git is too different from SVN and CVS.

The real problem was - someone had to work on the switch itself. There is so much tooling that had to be adapted to new version control. Everyone talked, but no one sent any patches. The same reason it is taking so long to switch from Subversion to Git.

I can't find it right now, but I've watched a video that explained CVS -> SVN timeline for FreeBSD. Here is "short" version of it: https://lists.freebsd.org/pipermail/freebsd-current/2008-Jun...

FWIIW, Matthew Dillon has converted DragonFly BSD from CVS directly into Git back in 2008, or, more specifically, on 2008-11-26:

https://leaf.dragonflybsd.org/mailarchive/commits/2008-11/ms...

It would appear that FreeBSD has switched the src repo from CVS into SVN on 2008-05-31.

Sure, Git surely must have improved since 2008, but did it really miss any really important features that have since been implemented back then, or is it just a matter of a much wider acceptance than anything else? It looks like $FreeBSD$ tags are now deprecated, because they aren't supported in Git; I recall these tags have always been the biggest blocker in any VCS considerations for both FreeBSD and NetBSD. TBH, I got the impression that they've never been properly supported by Subversion, either -- you had to install a FreeBSD-specific version of SVN for the tag expansion to work "properly".

I still think SVN is the worst out of all the VCS -- it has all the drawbacks of a centralised Git without many of the benefits, plus none of the benefits of CVS. For example, I wanted to have a local mirror on the history of the tree, which is absurdly easy to do with CVS with rsync -- you can even selectively ignore certain parts of the tree like --exclude "…src/contrib/???…/" --exclude "…src/gnu/…/…" (how do type a star symbol on HN?), but mirroring SVN was so complicated and inflexible that I had simply given up after it had used up way more space than the prior rsync mirror of CVS had ever required, and still wasn't even done. I then decided to entirely bypass SVN by directly switching to an "unofficial" git repo once the export from SVN back to CVS was completely deprecated and deleted at one point.

When FreeBSD switched from CVS to SVN, git wasn't able to handle repositories the size of the FreeBSD tree.

Significant improvements have been made over the past decade.

Is it really the improvements, though, or is just that Git has less resistance now in 2020 than it did in 2008?

DragonFly BSD has a smaller src, but it's been using Git since November 2008, converting away from CVS directly into Git, only a few months after FreeBSD moved from CVS into SVN in May 2008.

Dragonfly has a smaller tree and also a far less active one. Also, I think ports might have been a worse problem than sec -- not sure about that, just a vague memory.

Git has a steep learning curve especially when coming from long time svn users. At least that’s what happened to me. It took some mentorship to unfuck my brain and become acclimated to DVCS. Git is still hard to use in some circumstances.

> Git is still hard to use in some circumstances.

It's not like SVN was easy all the time either like some would like you to think.

https://stackoverflow.com/questions/3941291/a-sane-way-to-re...

Sure, we color our own experiences. All different. I think commit conflicts in svn are a lot easier than git conflict resolution.

I think you misread the issue; it's not a commit conflict. There was no second party. It's just a `svn mv` of a directory.

I agree that is anecdata, yet I've had 10y of extensive experience with subversion and 10y of extensive experience with git (plus a sprinkle of cvs and rcs to boot), both used on FOSS and company contexts. I've regularly teached both to beginners ans seasoned developers alike over the years. I've had the occasion to do things with both that would make many blemish and faint. Dismissing the counterexample with "everyone's different" is a bit hard of a handwave. I stand by my assessment: subversion looks easy on the surface but is fatally flawed down to the most simple actions, to the point of being able to cause data loss.

It also has to do with licensing around the tool itself (git(1))[0]. FreeBSD is licensed under BSD-compatible licenses, and will avoid reliance on tools that are licensed in an incompatible way. The advent of git-compatible alternatives[1] with BSD/ISC licenses make the move easier.

NB: the git format itself is not licensed, so only the tooling needs to have a "correct" license.

[0] https://github.com/git/git/blob/master/COPYING is GPL2

[1] https://gameoftrees.org/

BitKeeper has been available under Apache license for several years now. It's interesting that they didn't choose it. Not enough third-party use to justify adoption?

I can't imagine that OpenBSD's got has anything to do with FreeBSD's transition.

Replacing GPL'ed Git with a BSD alternative sounds interesting -- if not for anything else, then simply to combat the monoculture -- but assuming that developers are too stupid or stubborn to learn how to use a new VCS, and absolutely MUST have a `got checkout` in place of `git clone`, just because that's how CVS calls it, seems kind of silly.

Of course, you can't achieve world domination if you aren't any different, but having a brand-new `checkout` command -- is that really what's missing from Git?

I think the main barrier was the human factor. Workflows had to change and I imagine most contributors didn't want that.

https://wiki.freebsd.org/GitDrawbacks

This makes me sad. Subversion is such an elegant and simple versioning system. I really wish something would come along that does for git what subversion did for CVS.

Well, mercurial is kinda like a svn-flavored git

NetBSD was looking at mercurial last time I looked (with parallel implementations in mercurial and git), so hopefully they'll use it at least.

Does mercurial have something like git svn to bridge between so that others may use git protocol?

I'm getting tied of git and disappointed no one wants to get off of it instead more are using it.

https://www.mercurial-scm.org/wiki/HgGit

I agree, I still think that git vs mercurial was the VHS vs betamax of VCSs. Git was better for the kernel, but mercurial was better for basically everything else. Saner defaults, much more user friendly, better CLI and GUI, better docs etc...

Unfortunately as far as I'm concerned it's over. Git has won. It's just way too popular now, other DCVSs can't compete.

Don't fool yourself. It's only over if YOU think it's over. The Mercurial ecosystem is actually getting better and better.

Maybe got? It's still under heavy development but worth a look: http://gameoftrees.org/

Can you explain what got does that git doesn't do, or what its purpose is? I read through the FAQ [1], but I'm still very confused about its goal.

[1]: http://gameoftrees.org/faq.html

> Can you explain what got does that git doesn't do,

Probably nothing; if anything, it's a subset of git.

> or what its purpose is?

To be a saner subset of git with a more usable frontend.

Do you know if someone created a Linux-portable version yet?

Don't think so.

I am watching this one:

https://pijul.org/

Fossil has always seemed like a step up from `git` to me. Better defaults, a holistic solution, much more human-friendly interface and the format is better for working with code-wise as well, for querying and all that.

That being said it's not a drop-in replacement and no one uses it, so it's not as if you can enjoy how much better it is very easily.

https://fossil-scm.org/

NetBSD has been exporting into Fossil since 2011, it's really nice; in fact, the export into GitHub Git is done through an intermediary read-only Fossil repository:

http://wiki.netbsd.org/github/

https://src.fossil.netbsd.org/

Sadly, somehow the transition to the next VCS has been set to HG, which really makes no sense to me at all; Mercurial is not even BSD licensed, whereas Fossil is; plus, HG is just dead (even Bitbucket discontinued it); the decision to go HG in 2020+ really makes no sense, and I didn't even see anyone argument it; hopefully, it's reversed before it goes through.

Karl Fogel wrote a chapter for O’Reilly’s Beautiful Code on SVN’s interface that I go back to every now and again. Worth a read if you’re into code elegance:

https://www.red-bean.com/kfogel/beautiful-code/bc-chapter-02...

Bitkeeper?

I suspect interest in FreeBSD will rise with the demise of CentOS. What are the DevOps best practices for working with and deploying to jails instead of Docker?

Absent a good equivalent of Dockerfile, and compose features, I think this is doomed. (Longterm freebsd user btw)

What’s wrong with BastilleBSD?

I imagine the only thing wrong with it is that it's not nearly as well known as Docker.

Yes, mostly this. I think its targeting a different deployment model.

I am not using it but the features sound fantastic, like a complete network for every jail without complex docker network interface solutions.

Why dont they do a freebsd container runtime based on the jail semantics? They sound much more effective than the linux ones. Ok, the problem would be on how to run linux executables on freebsd.

Ok, the problem would be on how to run linux executables on freebsd.

That problem is mostly solved. Has been for years except for marginal sysctl() calls unlikely in datacenter usage of Linux.

Nothing, except it doesn't look to be doing additive composition on top of images and it doesn't look familiar if you were targeting pods in a k8s style deployment. As a thing in itself? Its fine.

If you have 5+ years investment in docker, the missing, parts here are going to hurt.

Mainly, what's missing is helpful evidence of transition.

E.g. I have a docker image i got from git which works as a one-shot execution chain to run xml2rfc and I got a version for cramdown which worked mostly the same way to do rfc/drafts from markdown. Where's the BastilleBSD equivalent? How do I translate a Dockerfile to a Bastille script?

There is a great simple framework that can ease a lot the deployment and management of Jails called BastilleBSD. You can install it with pkg install bastille.

https://bastillebsd.org/

They are vastly different creatures as you know, but in my experience on long-lived-but-must-keep-running services, fat-jails are a MUST.

OS package mismatches will bite you in the ass years down the road if you try to maintain thin-jails.

I manage alot of sites that Management refuses to invest any dev-time to upgrade old Python2.7 infrastructure. Having fat-Jails allows me to keep everything else upgraded.

pf is a fantastic firewall, but the syntax in OpenBSD is different then in FreeBSD.

VNET is worth it's weight in gold. Each jails has it's own network stack. That let's you avoid funky localhost workarounds.

iocage is a great framework for jails management (but it isn't necessary) you can do everything from jail.conf. Personally I prefer it for it's ZFS integration.

Hmm ... I was about to respond and say the opposite :)

My (personal) unbound jail has 95 files in it. The reason it has so many files is because it is simultaneously my 'nsd' jail.

Which is to say, I actually point two different 'jail' commands at the same directory - each with a different IP and command line:

  /usr/sbin/jail /jails/dns ns1 1.2.3.4 /nsd/nsd -c /nsd/nsd.conf

/usr/sbin/jail /jails/dns dns 1.2.3.5 /unbound/unbound -c /unbound/unbound.conf

So /jails/dns is a holder for two different dns servers and they run, jailed, from the same root. It's a little unorthodox but in this case I find it to be elegant.

I agree. I would run jails with very few files. A static executable, a config file, a log file, and some pipes maybe. Don't need anything else, don't want anything else (log file could be avoided with syslog, perhaps)

Bonus points: chflag schg the exe and config files such that they cannot be altered ... or better yet, make it a zfs filesystem and mount it read-only ...

I completely agree that the capability of running jails like that is awesome, and truth be told my original intention.

For some of my jails that is how I run them too. Especially where I am being especially paranoid about security. I was trying to answer about 'general use/unexpected gotchas' scenario.

We deal with the hands we are given. I wish I could run it as tight as that on all of my boxes, but the bosses I deal with do not want to waste money on upgrading old infrastructure but still get mad when it isn't running. So fat jails are my best compromise. My other option was a virtualbox/vmware setup.

Previous job, we were running almost everything in "normal" jails (as opposed to fat). We even stayed with ezjail, which is dead simple, because we were so comfortable with it and could understand how to manually fiddle with stuff when there was a need.

There are very very few things that cannot be told to bind to the real jail IP address instead of localhost. Most stuff will just work. And that's it. But there are also added benefits here:

* Parts of the filesystem cannot be altered, period. You update your base from your jailhost and every jail takes the change. That was an added security benefit.

* Pkg software can be individually installed per jail, goes into /usr/local. FreeBSD has already been brilliant in separating the base system from the third party software.

* For minor and major version bumps, we developed a simple set of steps and tools to upgrade a jailhost but leave the jails in their previous versions, maintaining multiple versions of base, like /var/jails/basejail104, /var/jails/basejail113, and so on. So the downtime is minimized, and each jail can be upgraded later in its own pace. This was a huge relief compared to having to upgrade everything only to find that some piece of software doesn't quite work correctly in the next version. And rollback is possible, by just copying the jail.

* Jails can likewise be very easily copied between jailhosts with a couple of rsync commands and some simple script.

Have used iocage and zfs and it is brilliant, but even without the benefits of zfs clone and snapshot, jails were (and I believe still are) a very pleasant experience to work with.

> I suspect interest in FreeBSD will rise with the demise of CentOS.

Indeed, the first thing I thought when seeing this post was "I wonder how long the support for each FreeBSD release is". Looking at Wikipedia (https://en.wikipedia.org/wiki/FreeBSD#Version_history), it seems to be around 5 years. Still no match for the 10 year support CentOS users expected; IMO, 5 years is too short when installing out of cycle (for instance, since the current stable 12.x was released two years ago, if I installed it today on a new computer I'd have only 3 years of support left).

This is much less of an issue with FreeBSD because the changes between releases are relatively small. Some systems out there had 20 years worth of upgrades, from v4 all the way to v12. I wouldn't like to try that on Linux.

You could probably get pretty close with Red Hat Linux 7.2 or 7.3 to 8 to 9 to Fedora Core 1 and on. There might be a break somewhere that the upgrade doesn't work, but nothing sticks out in my mind. Anaconda is a seriously impressive install/upgrade system.

> 5 years

If indeed 5 years suffices, Debian LTS may also be a candidate.

The reluctance towards moving to git in the FreeBSD community actually provided the central theme in the book "git commit murder": https://mwl.io/archives/2913

Hey, that book is about SkyBSD, not FreeBSD.

(Also, it's Warren Lash who gets murdered, not Warner Losh...)

I'm a little surprised they seem to be going with GitHub, I figured the FreeBSD folks would rather host it themselves.

edit Or perhaps not? This exists: https://git.freebsd.org/

IIRC the plan is to have a proper self-hosted Git repo, and mirror it on GitHub.

The link in the article is to discussion & documentation, not the destination for the code. In the big picture file[1] it says:

> We'll self-host the git repo and manage the first layer of mirrors ourselves. We'll publish the repository to github, gitlab and other places that are useful for our developers, users and contributors.

1: https://github.com/bsdimp/freebsd-git-docs/blob/main/big-pic...

Thanks I'd missed that.

Based on the migration docs, looks like they are hosting themselves. GitHub/GitLab will have mirrors.

Our company will never transition from SVN to Git because the transition tools fail on our ancient SVN repo. Learn from us: switch now because switching later may not be an option.

I migrated a 100k (20 years, 100Gb history) revision 20Gb workingcopy svn repo with full history and it was pretty painful. But in the end, so long as you can check out any revision (i.e. the repo is functional) then migration should also work, as should updating the repo format to the latest svn repo format. I assume the upgrade to a recent svn format is what's blocking? Even so, worst case it should theoretically be possible to do a looped checkout+commit from svn to git. Metadata such as commit dates can be rewritten later if needed.

You could also just keep the SVN repo around for its commit history and start a new git repo with the trunk imported without history. Sure you'll have to switch back and forth between the two repos for history, but over time it'll become less and less necessary.

Microsoft has done this for many long-lived repos; Windows and SQL Server are the two famous examples, with multiple such transitions.

Those weren’t using SVN, though, I don’t think. I can’t remember the name of the version control system they used, but it was some custom license of a proprietary system. Clear-case, maybe? Anyway, it was a nuisance, and I imagine the move to git was both a money saver and ultimately (once all of the migration pain and missing capabilities had been addressed) a nicer dev experience.

Source Depot, a fork of Perforce.

Yeah we decided against it and kept the history. Since the size grows quicker and quicker in an expanding team, cutting history e.g. to half would just cut a few percent of the size.

It actually works surprisingly well with git lfs now. Operations are quick enough and so long as you use persistent CI severs (so they never have to clone from scratch) builds aren't affected by the size either.

Brute force it; write a script to checkout every revision from SVN, checkin into git.

How did they do the conversion?

Looks like this is the repo for the scripts: https://github.com/freebsd/git_conv

which seem to be invoking this svn2git program (which is also copied into the freebsd/git_conv repo): https://github.com/svn-all-fast-export/svn2git

Great. Slowly approaching the features of more mature systems like DragonflyBSD.

You don't know what you are talking about.

Okay, I'll bite... what features does dragonfly have that freebsd lacks?

The src repository is on git. That's the only think I was talking about, and obviously in jest ;)

Supposedly better SMP support (this may no longer be true; benchmarks are several years old; but it may very well be).

HAMMER.

>HAMMER

No ZFS

Reply on news.ycombinator.com