Mobile App Rollbacks

My team and I are mostly former mobile engineers and we’ve felt the pain of app disaster scenarios in prod firsthand. No matter how extensive our test coverage, no matter how thorough our QA, no matter how much we put behind feature flags… our teams have shipped irreversibly-bad updates that break things and upset users.

And mobile hotfixes are slow. Your team has to triage the issue and assign it to the right team members, implement fixes (or carefully revert the offending changes), compile one or more RCs, upload to the stores, submit for review, then cross your fingers and pray to the store review gods for a speedy review. All of this takes time — hours in the best of cases, but potentially as long as days. And the negative impacts of even just a few hours with a critically broken app can be eye-watering: tens or hundreds of thousands of dollars in lost revenue, a flood of negative user reviews, and overwhelmed CX or ops folks.

So we decided to build a safety net.

Critical issue in prod? Roll back instantly. In the background of every normal release cycle, Runway automatically re-signs your last live production build and preps it for a possible rollback release — including preemptive submission for review (iOS only) — to remove all possible sources of delay. Should something go wrong in prod, there’s always a corresponding rollback build ready and waiting for single-click release in Runway.

Here’s how the rollback flow works in a bit more detail:

1. Your team is busy getting your next regularly-scheduled release out the door. Let’s call it version 1.2.0. 2. In the background, Runway takes your latest live build in production — say, 1.1.0 — and re-signs it, bumping the version to 1.2.1 and incrementing the build number as necessary. 3. Your team releases 1.2.0 as scheduled. Nice! The 1.2.1 rollback release is surfaced in Runway, just in case. For iOS, we also preemptively create a new version for the possible rollback in App Store Connect and submit the rollback build for review. 4. Your team is monitoring the rollout of 1.2.0 and identifies a critical issue You halt the rollout. 5. Right next to the button you used to halt the rollout, there’s another button to release the rollback build that’s ready and waiting. Click that, and you’re all set.

With rollback builds primed and ready to go, your team can address critical bugs in prod with a single click — cutting out all the lead time needed to investigate and implement a fix, get builds compiled and uploaded, and make it through the store review process.

Check out the blog post for more detail and visit runway.team to give Rollbacks a try.