GitHub Merge Queue Silently Reverted Code | Hacker News Reader

GitHub Merge Queue Silently Reverted Code

githubstatus.com

121 pointsmatthewbauer45 comments

The window to audit is 2026-04-23 16:05 to 20:43 UTC, about 4h 38m, and 'merged incorrectly' is unpacked in exactly no detail on the status page. Practical check is to compare the tree of each merge commit in that window against the tree of the rebased-and-tested commit that the queue ran CI on. GitHub Enterprise Cloud with Data Residency is still affected as of the latest status update, so DR customers shouldn't clear new queue merges yet. Would be kinda useful if GitHub published the specific failure mode, because content-drift and parent-linkage and dropped-changes all need different audit scripts.

Might want to check your git logs if you are using the Merge Queue feature. This afternoon, we found that Merge Queue was silently reverting changesets in the merge queue. Acknowledged by GitHub, but could be a very hard problem to debug if you aren't looking for it.

Same here, ffs GH.

4 people spent hours putting our repo back together at my company after this. GH has been unreliable and now they are breaking the core tenant of what I expect from this service.

> core tenant

tenet not tenant

This has caused a morning of fun for our team, how do you break one of the most fundamental bits of your system? Time to look at alternatives...

Self-hosting has been wonders. I thought I needed the social part of GH (i still kind of do, I like seeing what people are working on or liking) but overall its the same experience.

Imagine you pay someone to save your family pictures but sometimes they save only half your data. Instead they try to upsell you on AI filters for your pictures. GitHub, return to your core mission.

Tip: identify PRs with this problem by looking for discrepancies between the metadata (files/lines affected) for PRs via GitHub API and associated commits in the time window.

This was a pain to clean up!

I experienced a weird bug, where the github ci environment was checking out code that was throwing an error on build... the version tag definitely didn't have the code the error was coming from... I messed with the CI workflow yaml and finally cat'd the file in question... the error code was definitely there (2 lines repeated in the file) that was not in the repository.

I nuked the release tag and re-created it, and it started working... was just a really weird bug/experience.

Been self-hosting Gitea since ~9 months for my critical projects. Originally I did it because I didn't want AI to train on my private code but seeing what's happening to Github these days, it looks like a bonus that I don't face data integrity issues or uptime issues. I still use Github for my side-projects where I can directly push to main. I just hope these issues get fixed soon. Been using Github for many years now and I can't afford losing my Github history.

How did a bug like this get past their testing?

Given the push to AI/Copilot from MS I'd be impressed if they still did any testing.

CoPilot broke github actions in such a way that if too many requests to start a workflow came in at the same time it would reduce the capacity of the build server's queue until you did a full manual reboot of the build server and completely reset the queue.

Some people are saying they aren't doing testing, I know this to be incorrect in one very important way. When it comes to people, the ways in which a co-worker is going to be wrong are predictable. When it comes to AI written code, there is zero predictability as to what could be wrong.

Trying to test AI written code is like writing code with no try catch blocks, and expecting the application to not crash within 15 minutes of giving to the end user. They will try everything you didn't think to explicitly handle, and when it comes to AI there is no try catch block for subtly incorrect or incomplete logic.

They're at 88% uptime this year. I don't think they have testing.

Microslop®, unmatched quality.

Checked all my PRs that merged on the 23:d. Checked out each branch locally, rebased on trunk and ran 'git diff'. One of them had a diff which was the entire change that was marked as merged in the GH web ui.

This happened in complete silence and since the PR in question was just code refactor I would most likely never have noticed.

Considering switching to original method of simple git repo hosting and email patches like the kernel does it. A workflow I favor. Turns out there are no hosting services which provide such solutions. Lots of manual hosting setup and config required... Sad that GitHub sort of cast the mold for how git hosting has been done for the past 2 decades.

Github has been doing this for at least 5 years now. Merging a pull request merges whatever the latest commit on that branch is in the specific backend that handles the merge request. It does not merge what was just pushed into that branch.

for other folks currently in an incident trying to resolve the chaos this caused, the first commit we've found with issues in our repo is from ~10:30am pacific this morning

How long will users put up with this before they finally leave? Microslop is inventing new failure modes on a weekly basis.

Whats going on over there, is clippy in charge of the release button?

wow!

Wait what? How, what? This doesn't compute

Reply on news.ycombinator.com