HNHacker News
TopNewBestAskShowJobs

ethomson

3,773 karma · joined May 31, 2012

Product Manager at Stacklok; formerly product and software engineering at Vercel, GitHub, Microsoft.

https://www.edwardthomson.com/

Maintainer of libgit2. https://libgit2.github.io

[ my public key: https://keybase.io/ethomson; my proof: https://keybase.io/ethomson/sigs/48RVOIuAzrPKWNpOT1zlhLUUiT8VXFtqnuo5MzEds_w ]

submissionscomments
ethomson··on Microsoft and GitHub team up to take Git virtual file system to macOS, Linux
PM at Microsoft here. TFVC is completely different, though.

(To quickly clarify nomenclature: TFVC is "Team Foundation Version Control", the name of the centralized version control system in TFS and VSTS, which are our on-premises and cloud hosted development platforms, respectively. They both support both TFVC and Git - Git with GVFS, and they do more than just version control, hence the naming clarification. Apologies for all the three letter acronyms.)

Anyway, TFVC is totally different and a centralized version control system with all the workflows that go along with it like expensive branching. The desire to move to Git opens up all the awesome workflows that Git offers.

The impetus here was to standardize the entire company on a modern set of development tools - one engineering system across the company - hosted in Visual Studio Team Services. We could have moved them to TFVC, it's remarkably similar to its predecessor, "Source Depot" which is a Microsoft internal tool that the Windows team was using.

But that's a lateral move. There aren't really any benefits to TFVC over Source Depot except that we sell and support one to end users and don't with the other. So that has some nice organizational benefits but not enough to warrant moving everybody and the build/release farms over.

But moving to Git unlocks all sorts of new workflows - lightweight branching, pull requests, all that good stuff that everybody who uses Git is totally used to.

Anyway, that's the background on Git vs TFVC. And, no, I totally agree that Git not supporting giant monorepos is not a failing, per se. But it is a limitation, and thankfully one that we're working to help overcome.

ethomson··on Microsoft and GitHub team up to take Git virtual file system to macOS, Linux
PM for Git at Microsoft here. We explored splitting it up. It's a 300GB repository and it's been a monorepo for the last 20 years. Splitting it up logically would take a lot of time (to put it mildly) and development would stall while we did it. And once we did, we would have 100 3.5GB repositories? 3500 100MB repositories? Neither of these are particularly appealing and getting changes checked in atomically across multiple repositories is insanely challenging. There's no doubt that we would need to build tooling to make this work for us. (We did actually explore this direction, but ultimately decided that it would be too much work for too poor an experience.)

Instead, we decided to - as you put it - change Git to fit our needs.

ethomson··on Microsoft and GitHub team up to take Git virtual file system to macOS, Linux
git-lfs supports locking, beginning in 2.0.
ethomson··on Ask HN: Who is hiring? (November 2017)
Microsoft Visual Studio Team Services | Software Engineers, Engineering Managers and Program Managers | Redmond, WA and Durham, NC | Onsite

Visual Studio Team Services (VSTS) provides software development teams with version control (Git and centralized), build automation, agile work management and social experiences in the cloud.

VSTS is built using Azure (SQL, Storage, Service Bus, Active Directory), .NET/C#, ASP.NET MVC, Redis, JavaScript, TypeScript, jQuery, Knockout, and more. We ship new experiences at the end of every sprint (every 3 weeks), and we use VSTS to build VSTS which allows us to use everything before our customers.

We have open roles for Software Engineers, Engineering Managers and Program Managers: http://aka.ms/vstsjobs

ethomson··on Exploding Git Repositories
It's not security through obscurity. It's defense in depth.
ethomson··on Exploding Git Repositories
As in a tree that points to itself? You cannot, since a tree would have to point to its own SHA1. So this would require you to know your own tree's SHA and embed it in the tree.
ethomson··on Exploding Git Repositories
Yes, hosting providers need rate limiting mitigations in place. GitHub's is called gitmon (at least unofficially), and you can learn more at https://m.youtube.com/watch?v=f7ecUqHxD7o

Visual Studio Team Services has a fundamentally different architecture, but we do some similar mechanisms despite that. (I should do some talks about it - but it's always hard to know how much to say about your defenses lest it give attackers clever new ideas!)

ethomson··on GitLab raises $20M Series C round led by GV
No - although I am an aging unix neckbeard, I do not assume that you _need_ that to be successful.
ethomson··on GitLab raises $20M Series C round led by GV
Yes, this is true to an extent. But price is one of many factors that go into GitHub's design decisions for infrastructure planning. (I used to work on the Git Infrastructure team at GitHub, which is responsible for exactly this.) Nor is it the _first_ consideration. Price is important (obviously) but only after performance and resiliency.

Putting repositories on bare metal on-premises makes sense for GitHub's architecture. They run git itself, which is happiest packbuilding with as many IOPS as it can manage. But they've also built out a number of custom performance monitoring and load balancing applications, and any single Git repository is replicated across a number of servers using spokes, their custom Git load balancing system. They run on-premises because they've done a detailed analysis of their needs and hired a staff of developers and infrastructure experts to deliver it. And as a result, they're the world's largest Git hosting provider.

But you _can_ be perfectly happy in the cloud.

Visual Studio Team Services hosts its Git repositories in Azure. Since VSTS didn't start out hosting Git repositories, but Git repository management was added _after_ it already had a cloud-first architecture, it didn't make sense to host repositories on-disk. (Nor would relying on Git for Windows have been performant five years ago.) Instead, VSTS built out custom Git server implementation to take advantage of storing repositories in SQL Azure and Azure Blob Storage. And this implementation scales up to the needs of Microsoft's customers - both external and internal - who are hosting the world's largest Git repositories.

So I think that you can be happy on-premises _or_ in the cloud. I think it comes back 100% to your point that you need to hire for that expertise.

ethomson··on Ask HN: Who is hiring? (October 2017)
Sorry to hear that. I'm not a recruiter, but I'm told that the fall recruiting season is a particularly busy time for them and we receive a very high volume of applications and resumes and that it's not inappropriate to follow up.

Of course, we do keep your application on file in case an opportunity that matches your skills and interests becomes available.

ethomson··on Ask HN: Who is hiring? (October 2017)
Microsoft Visual Studio Team Services | Software Engineers, Engineering Managers and Program Managers | Redmond, WA and Durham, NC | Onsite

Visual Studio Team Services (VSTS) provides software development teams with version control (Git and centralized), build automation, agile work management and social experiences in the cloud.

VSTS is built using Azure (SQL, Storage, Service Bus, Active Directory), .NET/C#, ASP.NET MVC, Redis, JavaScript, TypeScript, jQuery, Knockout, and more. We ship new experiences at the end of every sprint (every 3 weeks), and we use VSTS to build VSTS which allows us to use everything before our customers.

We have open roles for Software Engineers, Engineering Managers and Program Managers: http://aka.ms/vstsjobs

ethomson··on The tale of aux.c
Length was, maybe. Reserved device names (like CON, AUX, etc) are still reserved.
ethomson··on MacOS 10.12.6 Source
Yes, I'm very aware of our license.

It allows people to link it in their non-free, proprietary software without any vitality on that proprietary software. It still requires them to make the source for the version of libgit2 that they are using available upon request.

ethomson··on MacOS 10.12.6 Source
I'm surprised to see you downvoted here. As a maintainer of an open source project, libgit2, which is GPL licensed and used by Apple, I am rather frustrated by Apple's behavior.

The source for libgit2 itself is not present on their opensource site, though they use it in Xcode. (Git itself is present, but is sometimes out of date compared to what they're actually shipping.)

I've emailed them about this and they offered to provide me the source they used. This feels like they're trying to do the bare minimum of compliance - and making even that nonobvious (since they have an obvious distribution mechanism.)

This complaint is not a claim that they're violating our license, but it's definitely not an ideal situation, and I'm sympathetic to your frustration.

ethomson··on Atlassian: Our bold new brand
Visual Studio Team Services. I'm not sure what you mean by "massive" but VSTS is the only service that handles the truly massive projects, like Microsoft Windows. https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-large...

Edited to add ObDisclaimer: I work on the VSTS team.

ethomson··on Bats slam into buildings because they can't 'see' them
The post explains this: they don't. Standing water exhibits the same behavior as reflectiveless flat surfaces.

When they fly over a space and "see" nothing beneath them, they interpret this emptiness as a flat, smooth body of water.

Thus the only real problem here is that for flat horizontal surfaces that aren't water, the bats will try to drink from them.

ethomson··on Introducing Exclusive Features to GitLab.com Bronze, Silver and Gold Plans
If you're concerned about performance, Visual Studio Team Services hosts the world's largest Git repositories. (The largest being Windows itself.) And has to stay performant while doing it.

Indeed, most people don't need to host repositories that large but we're proud that our performance scales out even to repositories that are massive.

(ObDisclaimer: I work on VSTS.)

ethomson··on Ask HN: Who is hiring? (September 2017)
Microsoft Visual Studio Team Services | Software Engineers, Engineering Managers and Program Managers | Redmond, WA and Durham, NC | Onsite

Visual Studio Team Services (VSTS) provides software development teams with version control (Git and centralized), build automation, agile work management and social experiences in the cloud.

VSTS is built using Azure (SQL, Storage, Service Bus, Active Directory), .NET/C#, ASP.NET MVC, Redis, JavaScript, TypeScript, jQuery, Knockout, and more. We ship new experiences at the end of every sprint (every 3 weeks), and we use VSTS to build VSTS which allows us to use everything before our customers.

We have open roles for Software Engineers, Engineering Managers and Program Managers. See the options and apply at http://aka.ms/vstsjobs

ethomson··on A pilot explains turbulence
I can't recommend Patrick Smith's writing highly enough. I used to be somewhat uncomfortable when flying: the lack of control was extremely unsettling. But after reading Ask the Pilot for a while, I came to understand and respect the role of the pilot.

I would also strongly recommend his books, _Ask The Pilot_ and _Cockpit Confidential_ for anybody who travels (and especially for nervous fliers). Now I fly quite regularly, and love it.

ethomson··on Critical security updates for Git, Subversion and Mercurial
I'm not familiar with any that exec ssh. Certainly libgit2 does not, nor do the language bindings on top of it. So libgit2, LibGit2Sharp, Rugged, Jagged, NodeGit are unaffected.
ethomson··on Git v2.14.0 released
Not just for Mac, there's a Git Tower for Windows as well.
ethomson··on Ask HN: Who is hiring? (August 2017)
Microsoft Visual Studio Team Services | Software Engineers, Engineering Managers and Program Managers | Redmond, WA and Durham, NC | Onsite

Visual Studio Team Services (VSTS) provides software development teams with version control (Git and centralized), build automation, agile work management and social experiences in the cloud.

VSTS is built using Azure (SQL, Storage, Service Bus, Active Directory), .NET/C#, ASP.NET MVC, Redis, JavaScript, TypeScript, jQuery, Knockout, and more. We ship new experiences at the end of every sprint (every 3 weeks), and we use VSTS to build VSTS which allows us to use everything before our customers.

We have open roles for Software Engineers, Engineering Managers and Program Managers: http://aka.ms/vstsjobs

ethomson··on MySQL infrastructure testing automation at GitHub
You're technically correct, which is of course the best kind of correct. But it's not quite so easy.

I can't speak for any other Git hosting service, of course, but if you have a private repository in Microsoft Visual Studio Team Services then:

- all engineers are given a background check before being hired - very few engineers have access to production servers or databases, only those who have an operational neccesity - engineers that do have production access are given a still stricter background check

Certainly we don't have private repository just lying around, and yes we do treat them like they're your financial data.

Microsoft hosts all of its source code in VSTS, including Windows. This is data that we absolutely want to protect. Our customers' data is no different.

ethomson··on On Conference Speaking
Personally, I love speaking and presenting. I'm not bored giving a talk several times, they're different each time, even when you think it'll be the same. Having to compress something down to 30 minutes or extend something to 60 makes a different, plus the things that I've been thinking of that day. I also often try to tie in my talk with a prior talk or with the keynote.

I also don't find the rehearsals boring; in fact, around 24 hours before, I'm already starting to get excited about the upcoming talk. So I'm pretty amped up.

One challenge is multiple talks back-to-back. It's hard to keep that momentum up for several days in a row. I've definitely had occasion to give a great, high-energy talk and then give a talk a day or so later that just feels "off", especially in terms of my energy level.

ethomson··on On Conference Speaking
On the whole, this is excellent advice. The introduction is completely true for me: I give many talks every year and it is, without question, a _lot_ of work. I suspect that everybody is a bit different as to how they prepare, but like the author, I do the cold rehearsal in my hotel room a half-dozen times (at least) before I actually go to give the talk.

I also break my talk into logical chunks - say five or six sections. I practice each of those individually, timing them. This gives me an average for how long each section takes, so I have a schedule written down. This lets me know how far over or under my time allotment I am so that I can adjust on the fly, either adding some additional explanation to some areas or subtly truncating something.

I always know my "bail out" slide - if I end up running out of time, what's the "thank you!" slide number? If you simply type in that slide number in PowerPoint or Keynote, it will jump to that slide without fanfare. Don't ever tell your audience that you ran out of time to get to all your material, or flip through the slides to the end that they won't get to see. They'll feel like they were ripped off. (Also, make sure to structure your talk so that the special bonus material is at the end, so they're _not_ actually ripped off.)

ethomson··on The largest Git repo
300 GB is not the size of _the repository_. It's the size of the code base - the checked out tree of source, tests, build tools, etc - without history.

It's certainly possible that somebody created a 1 TB source tree in Git, but what we've never heard of is somebody actually _using_ such a source tree, with 4000 or more developers, for their daily work in producing a product.

I say this with some certainty because if somebody had succeeded, they would have needed to make similar changes to Git to be successful, though of course they could have kept such changes secret.

ethomson··on The largest Git repo
CVS and SVN are probably a bit better at scaling than (stock) Git. Perforce and TFVC _are certainly_ better at scaling than (again, stock, out-of-the-box) Git. That was their entire goal: handle very large source trees (Windows-sized source trees) effectively. That's why they have checkout/edit/checkin semantics, which is also one of the reasons that everybody hates using them.

GVFS intends to add the ability to scale to Git, through patches to Git itself and a custom driver. I don't think this is a hack - by no means is it the first version control system to introduce a filesystem level component. Git with GVFS works wonderfully while offline for any file that you already have fetched from the server.

If this sounds like a limitation, then remember that these systems like Perforce and TFVC _also_ have limitations when you're offline: you can continue to edit any file that you've checked out but you can't check out new files.

You can of course _force_ the issue with a checkout/edit/checkin but then you'll need to run some command to reconcile your changes once you return online. This seems increasingly less important as internet becomes ever more prevalent. I had wifi on my most recent trans-Atlantic flight.

I'm not sure what determines when something is "a hack" or not, but I'd certainly rather use Git with GVFS than a heavyweight centralized version control system if I could. Your mileage, as always, may vary.

ethomson··on The largest Git repo
Microsoft's tooling is Windows only _today_. GVFS is open source _and_ we are actively hiring filesystem hackers for Linux/macOS.
ethomson··on A look at my BBS Software from '93
That's funny - I have always simply typed `ethomson@` as my password for anonymous FTP, without ever specifying the hostname portion. The then-popular wu-ftpd had an option to enforce that there was a `@` in the password for anonymous access, but it didn't verify anything after that.

I read somewhere that therefore a reasonable shorthand was to drop the hostname portion of your address since the FTP server knew your source hostname anyway. So this is similarly etched into _my_ muscle memory, but looking back I have no idea where I read this or if it was actually common at all or if I was the only one.

ethomson··on A look at my BBS Software from '93
What a blast from the past. I remember TeleGrafix fondly; if I'm being honest, I didn't really use RIPterm all that terribly often; I've always felt more at home in text. But I did run a BBS (1:233/24) and I enjoyed hacking on it more, I think, than actually running it. So I spent a lot of time trying to shoehorn RIPscrip into Maximus, more for the novelty than for any actual demand on the part of my callers.

After googling, I just realized that Maximus 3.0 eventually included RIPscrip support built-in, but I had lost interest in my BBS and moved on to hacking on Unix systems by then.

But I remember RIPscrip being a nice format. Thanks for providing me with many an afternoon spent tinkering with it.

← PreviousPage 4 of 12Next →