Comment by lapcat | Hacker News Reader

Comment by lapcat | Hacker News Reader

Parent Full thread

> How is it that simultaneously, so many apps fail review for tiny details, showing that apps really are having manual review done on them...

> ...and yet all this spam is getting through?

It's important to understand that App Review is not consistently nitpicky but rather randomly nitpicky.

Every indie dev has stories of urgent bug fix updates getting rejected for stupid, unrelated, nitpicky reasons, things that had already been in the app for months or years. Yet every indie dev also has stories of major updates flying through review in minutes, so fast you know it's impossible that anyone looked carefully.

I do get the impression that they're doing random inspections.

"Every week, over 500 dedicated experts around the world review over 100K apps." https://www.apple.com/app-store/

Apple seems to be bragging about that, but if you estimate how much time is spent on each app, the numbers look pretty bad. If 500 reviewers spend full-time 40 hours every week doing nothing but reviewing apps — no breaks, no meetings, no training, no vacations, etc. — that leaves only 12 minutes per app, maximum. And they're reviewing not just the app but the App Store metadata: release notes, description, screenshots. They definitely do metadata rejections.

> I do get the impression that they're doing random inspections.

As someone who's been responsible for various iOS App Store submissions for closing in on a decade, the impression I get is that the variation is primarily due to inconsistencies in process between reviewers. The corners probed by one reviewer might go entirely ignored by another and what one reviewer finds perfectly fine constitutes a violation in the eyes of another.

The Play Store review process on the other hand comes off as feeling almost entirely automated barring a handful of exceptions, which works fine until some innocuous change in code or the automated review system trips something somewhere and your app listing is removed. Good luck getting in touch with a real person to fix it.

I mean there’s that too. Inconsistency and lack of resources are both present.

> bug fix updates getting rejected for stupid, unrelated, nitpicky reasons, things that had already been in the app for months or years

This reminds me of code review in general. Some reviewers have the etiquette of "if it was there already, then it's outside the scope of this review," and some don't.

Maybe a reviewer should have two responsibilities:

1. Review the proposed _change_ and verify that it meets all criteria and doesn't introduce defects.

2. Make note of any existing defects, which are then recorded.

Maybe an app developer can be required to resolve everything found in (2) before the _next_ review.

I've never released an app before.

> Some reviewers have the etiquette of "if it was there already, then it's outside the scope of this review," and some don't.

> Maybe a reviewer should have two responsibilities: [...]

For code review: you can make lots of different choices for responsibilities and still have a good code review process. It just needs to fit your goals and people in your organisation should be on the same page about it.

So for example for code review, I don't think the reviewer should try very hard to understand the change. Explaining the change so that it is easy to understand is one of the responsibilities of the author who proposes the change.

Similarly it's upon the author to demonstrate that the proposed change doesn't introduce any defects and to explain how it meets all criteria. It's not on the reviewer to go bug hunting.

That is so that the next guy who view the change in a few years in version control history to understand how the software developed (or how a bug was introduced) has a fighting chance to understand what happened from what was preserved in version control alone.

1. Since most changes are compiled binaries, you would have to rely on the submitter to self-report what changed.

Things open source developers say because they don’t have experience with binary analysis. Not to say that Apple’s reviewers are going to be looking at IDA or Ghidra dumps, but they can see added capabilities, frameworks, symbols, or assembly and call graph changes.

Not that it matters, because they don’t have the time to manually review source code changes either. At best there’s likely some automated static analysis for undocumented API symbols and common malware signatures.

A larger issue is that to review these kinds of changes you need to have a baseline competency in understanding how code works, and that’s expensive to procure at the scale needed to do proper review.

A bad actor can just carefully exclude the reviewers from ever seeing those changes (and it's routinely done on a large scale).

Code review is typically based on a premise that the person submitting the change is acting in good faith. This is the only reason it works, because otherwise it’s difficult to review them sufficiently. App review must consider developers to be potentially hostile.

> App review must consider developers to be potentially hostile.

And vice versa. ;-)

Generally I’d plead incompetence but if you’re trying to compete with Apple I’ll accept your point ;)

Whenever I get a comment about a pre-existing bug in a code review I make a ticket to fix it and thank them for pointing it out, but I never fix it in the PR. It gets too messy for the other reviewers and too confusing for QA when you end up with a bunch of unrelated fixes in one update.

> It's important to understand that App Review is not consistently nitpicky but rather randomly nitpicky.

This has been my experience as well. A long time ago, my company had an iOS communications app that we'd occasionally white-label for different clients (about 7, in total). The only changes to the apps were the color palette and the name/logo that appeared on the login page.

For whatever reason, we were never able to release the white label for the 4th client. All other versions sailed through review, with the occasional holdup because the terms of service web page linked in the app was part of a web site which, through its navigational elements, would allow one to sign up for our service online. This is against Apple terms if you don't also allow sign up in the app. We were usually able to get this resolved because it wasn't actually a sign-up form, but a form to request more information from our sales team.

For that 4th white label, that "sign-up page" became an issue we could never resolve. Our app was for enterprises, so a per-user purchase/sign-up in the app didn't make sense. We tried to move the terms of service and other required pages to their own web site with no links to anything on our company web page, but that was disallowed because it didn't link to our company web page. Apparently the only allowed solution was remove the request info page from our corporate web site.

In the end, we simply resubmitted as a new app and it was approved without issue.

12 min per app should be plenty, but they should be rejecting a bunch of this garbage in about 20seconds.

Reply on news.ycombinator.com