ethomson

Yeah - the hybrid cloud and on-premises build agents is actually one of my favorite features. The open source project that I help maintain (libgit2) uses VSTS's hosted Linux, Windows and macOS build agents, but add support for some other platforms (like ARM) with on-premises build agents.

("On-premises build agents" makes it sound fancier than it is, it's actually Raspberry Pi's on my desk, but hey, whatever works, right?)

You're right that we need to provide some better comparisons here, but the big reason that I'm excited about making individual components easier to adopt isn't for competition, it's to help people round out what they're missing in a dev tools stack.

There's a lot of shops that are happy with their git hosting provider and their agile planning tool and go looking for a CI/CD tool and may have been turned off by VSTS because it looked like a monolith. Making it clear that you can just use Azure Pipelines without having to use the entire stack was a priority for us, but you're absolutely right that we need to round this out with some better case studies. Thanks for the feedback.

PM for Azure DevOps here (formerly VSTS). It is a rebranding, but it's more than merely a rebranding. We're breaking out the individual services so that they're easier to adopt. For example, if you're just interested in pipelines, you can adopt only pipelines.

We've got some neat new features launching today and more coming soon. My favorite new feature is the Azure Pipelines app the the GitHub Marketplace. It makes it one-click to build a continuous integration and pull request validation pipeline right from your GitHub repository, you don't even need an Azure DevOps account to get started.

PM for VSTS here. The final scale unit in South Central US was brought back online a few hours ago, which means that the final accounts that were affected should now be operational. We're still restoring package management to some accounts, but otherwise, you should be back to working. Please feel free to reach out to me if you're still having trouble. Email is my HN username @ microsoft.com. We're _very_ sorry for this very significant outage.

As a Microsoft employee, I use Mac, Linux and Windows regularly. I write code in vim. I write quick-and-dirty scripts in Perl or Ruby. I’ve shipped code in Java, C and C#. I browse the web in Chrome. I push to GitHub and VSTS.

Microsoft doesn’t force employees to use Microsoft tools; Microsoft forces employees to think critically, empathize with our users and build great experiences for them.

The Git community is great because even though many of the interested parties compete with each other in some form or another, we always put that aside. And that's especially true for security issues.

Thanks, peff, for the feedback. I pushed some changes to try to clarify that this does indeed require `clone --recursive`, and I added a note to credit Etienne Stalsman explicitly. That was an oversight in my haste.

Thanks for catching that, I've updated the blog.

That's a really interesting question. I tried to think about that when I was writing this blog post and couldn't really think about anything. Perhaps setting some permissions on a file or running `chattr` / `setfacl` is the best idea I had, but I omitted any suggestions because I've never actually used a post-checkout hook in anger.

Part of this bugfix includes changes to "git fsck" to reject these problematic repositories. Git hosting sites like GitHub and Visual Studio Team Services are actively blocking these repositories today to prevent being used as vectors for propagating this vulnerability.

It's not that you can be compromised by running the code contained in a git repository, it's that there's a bug that allows somebody to create a git repository that will cause you - merely by (recursively) cloning the repository - to run arbitrary code that they've provided.

This is, of course, unexpected. And while you should perhaps raise an eyebrow if somebody you don't know asks you to recursively clone a repository that you're not interested in - this is indeed a problem and you should upgrade your Git client.

A symbolic ref is a ref that points to another ref instead of a ref that points to a commit. `HEAD` is a symbolic ref. (It should be your only symbolic ref.)

Well, then I wish Mercurial nothing but luck in winning the lucrative free version control client market.

Edit: sarcasm aside, you're right in saying that there is competition here. And competition is healthy, it's what drives us to solve problems in different ways and _allows_ us to collaborate. But to suggest that we shouldn't use each others technologies feels short sighted and like the wrong kind of competition.

Git Community PM at Microsoft here: I'm very impressed by Facebook's source control team in general (and Mononoke in particular). They're doing a lot of interesting things, and they've developed a very opinionated workflow about working in their monorepo that is very helpful for their workflows.

Another comment suggests that there's "fierce competition" in the version control space. In fact, I think that there's fierce collaboration in the version control space! Durham Goode from their team gave an interesting talk at Git Merge 2017 about how they're using Hg and some of the interesting user-experience tools that they've built on top of it: https://m.youtube.com/watch?list=PL0lo9MOBetEGRAJzoTCdco_fOK...

I don't personally prefer Mercurial (I like Git, as you might have guessed), but I think that there's some opportunities to learn from their user experiences and perhaps add or improve some high-level Git commands.

I think that the message it sends is that we collaborate with each other - Mercurial and Git, Facebook's Mononoke and Microsoft's GVFS. And since GitHub is _the_ place where open source software development happens, that's where this is hosted.

We're not enemies. We're not competing. We're trying to make developers lives better, and there are a number of techniques that can work, and a number of tools that can help them.

It is not a finite list of rules. It is an explanation of the values (eg, respecting diversity) and a list of examples of things that are obviously prohibited. It is explicitly "not an exhaustive list of things that you should not do", and although it does list some _examples_ of things that are prohibited - it provides examples that are "not limited to" the ones provided.

This clearly reduces the ambiguity in the community. And even if some ambiguity does remain - which, as you point out, is inevitable - having examples of obviously impermissible behavior and a framework for conflict resolution is requisite for a community.

I'm not sure why it matters how long we've been using FreeBSD, but I've been using FreeBSD off and on since roughly 2.1.

You may be shocked to learn that some people actually do not have common sense. And that there is very little recourse when those people - the ones lacking common sense - decide to harass or intimidate people. Unless you actually _make rules_ about your community, you cannot _enforce those rules_.

So there are no "common sense" solutions to these problems unless you actually spell them out in advance. As an open source maintainer myself, I assure you that we do not do these things "to please some groups" but to have a framework for operating when people who lack "common sense" decide to be jerks and harass or intimidate other members of our community.

Jekyll for my podcast's web site (https://allthingsgit.com/). I became a fan of Jekyll when I worked at GitHub, but I don't use GitHub Pages, I needed to scale it larger. (edit: in particular, I needed something that can host audio and video.) So my workflow is GitHub for hosting, and Visual Studio Team Services for CI builds. It monitors the GitHub repository and on commits to master, it runs Jekyll in a docker container. It then deploys to Azure, where I host the static web site. (I use Azure CDN for hosting the audio episodes.)

I would use VSTS for the whole pipeline, including the Git repository hosting, but I wanted to have a public repository, in case listeners wanted to submit pull requests. So far that hasn't happened, so I'm missing out on some of the VSTS features, but maybe it still will.

I also have a weekly newsletter (https://developertoolsweekly.com/) that uses a static site generator that I build myself. It's very Jekyll-inspired and uses Liquid templates, but instead of generally processing markdown, it takes a simple list of links and descriptions and emits a website as well as the HTML and plain text for the emails.

This workflow is similar, the generator is run in a Docker container, but this workflow is entirely VSTS and Azure.

All the tools that have been mentioned already and Visual Studio Team Services, which offers agile project planning, unlimited private git repositories (free for up to 5 users) and hosted CI/CD pipelines. http://visualstudio.com/team-services

For better or worse, I spent more time giving talks than watching talks in 2017.

My favorite talk that I watched in 2017 was Durham Goode's talk at Git Merge 2017, concerning scaling Mercurial at Facebook. It's always good to share knowledge about version control and scar difficulties:

https://m.youtube.com/watch?list=PL0lo9MOBetEGRAJzoTCdco_fOK...

My favorite talk that I gave was at GitHub Universe. I spoke about how Microsoft adopted the Git version control system and - ultimately - open source:

https://m.youtube.com/watch?v=LXu80xXwFY0

Where in Heathrow are there digital gates to exit?

The US already has a variety of digital gates to _enter_ the country - similar to the digital gates that Heathrow has at immigration - but this article is talking about special electronic gates at _exit_ which I've never seen at any LHR terminal.

My "commits per quarter" graph jumps from Q4 2015 to Q1 2017, without showing any data for the quarters of 2016. That's interesting because I moved from the US to the UK, which was quite a shock! It took me a while to really feel comfortable in my new home, especially since I had just taken a new job and it too was changing a lot. I guess I really didn't really "fit in" at the new job. Upon reflection, I guess 2016 was a bit of a lost year for me.

I'm impressed. This visualization is _really_ advanced.

Or maybe the labels are just incorrect.

True. Source Depot originated as Perforce and is a fork; it had (at one time) a pretty large team of developers working on it and it has been heavily, heavily modified in the many years since.

It's not impossible but it too is impractical for performance reasons. Providers that run git core repack repositories regularly because packfiles are efficient; loose objects are horribly inefficient - even on the local filesystem. Moving them to a network filesystem is impractical.

GitHub only shares objects among forks. Source: I used to work on GitHub's Git Infrastructure team, but this is publicly available information.

You can read about their architecture in their discussions of Spokes, which replicates repository networks (the original repository and its forks) across data centers. eg: https://githubengineering.com/stretching-spokes/

Trying to put all of GitHub's object files in a single packfile - even just putting them on a single server - would be impossible.

But even on hosting providers with a bespoke implementation - that do not use core git to manage Git repositories - this would be challenging. We have a custom Git server implementation in Visual Studio Team Services, but it still makes sense to shard object storage with the repository: you have to worry about scalability and performance, but also things like data sovereignty. We can't just put a user's git repository in some global SQL Azure database that contains all the repositories in Visual Studio Team Services: the repositories need to be geographically located with the VSTS account they created.

Can you clarify what you mean by "more close to the online service"? We update TFS with the bits from VSTS pretty regularly; are you looking for a faster cadence or am I misunderstanding what you're saying?

I'm not the right person to ask about Windows organization, I'm afraid. But I will say that the nice thing about moving to Git is that its lightweight branching makes refactoring much easier.

I agree that these sentences gloss over the details - and rightly so: it's an article in Ars, after all, not a journal article. We (Microsoft) have published a lot[1] about our journey to put Enterprise-scale codebases, like the Windows codebase, into Git. I think you'll find that they expand upon some of the things that are glossed over.

And I would encourage you to read them, so I won't belabor the details here, but to give you just one example: git blame is not the issue that we're most concerned about. The basic, highest priority functionality in Git - things like git add - are the the issues we're most concerned about.

git add is O(n) on the number of files in the repository. When you run `add`, git reads the index, modifies the entry (or entries) that you're changing, and writes the index back out.

The Windows repository has about 3.5 million files[2]. Running `git add` - when we started thinking about moving Windows into Git - took minutes.

Can you imagine running `git add` and having it take 10 minutes?

Now obviously there are some inefficiencies here - there's quadratic operations and the like that went in assuming that nobody would ever put 3.5 million files into a Git repository. And we've been cutting those out over time[3].

Thankfully, Git does have some functionality to support very large repositories - using shallow clones, sparse checkouts and the like. We've added narrow clones, to only download portions of the tree, and automation to handle this automatically without user intervention.

That's the scaling work that we're doing with GVFS. And these changes bring the P80 time for git add down to 2.5 seconds[4]. We've been contributing these changes back to git itself, and we're thrilled to work with industry partners like GitHub who are also interested.

Sorry to go on - version control is a passion of mine, as it is for many of us working on Visual Studio Team Services. Your conclusion is very much correct: Git wasn't designed for this from the beginning. Thankfully, software isn't immutable, so we're scaling it up.

[1]: http://gvfs.io/

[2]: https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-large...

[3]: https://blogs.msdn.microsoft.com/devops/2017/05/30/optimizin...

[4]: https://blogs.msdn.microsoft.com/devops/2017/11/15/updates-t...

We were using a mix of tools throughout Microsoft: several teams were using Source Depot, which is an internally developed centralized version control system. It was built to handle large teams, like Windows and Office. It was the precursor to Team Foundation Version Control, which is the centralized version control system available in TFS and VSTS, which is also capable of scaling to large projects, and many organizations within the company were (and are) using that.

GVFS is part of our effort within the company to standardize on a single set of best-of-breed tools, and use the same tools inside Microsoft that we deliver to customers. So we're adopting one engineering system throughout the company and we're moving everybody to Visual Studio Team Services.

Microsoft PM here. It's important to note that Hg also had similar constraints. So Facebook was in a position of having to fix Git or Hg to suit their needs. It turns out that for a variety of factors, Hg was the better choice for them to hack on.

Microsoft took the same look and went the opposite direction, and decided that Git was easier for us to hack on. This was a no brainer for us. We build multiple tools that host peoples Git repositories (TFS and VSTS) and have Git repositories as deployment mechanisms for Azure. We have several contributors to tools in the Git ecosystem on staff, including people contributing to core git, and the maintainer for Git for Windows, libgit2 and LibGit2Sharp. But we have comparatively little institutional knowledge of Hg.

That post you linked is awesome. Facebook has done a lot of really impressive work scaling Hg and a lot of the lessons learned at Facebook and implementations in Hg are very similar to what we've done with Git.

This video from Durham Goode on Facebook's version control team (from Git Merge, amusingly) is also awesome: https://m.youtube.com/watch?v=gOVD-DrUpwQ