IOS5: There's a reason it's called 'beta' software
mbarclay.net
mbarclay.net
*
Title: Users leave bad reviews for app that don't work on beta OS releases
Summary:
The point of the prereleases is to give developers time to modify their apps to address any compatibility issues (as well as add new features) before the public release of the operating system. Despite Apple's best intentions at keeping OS betas limited to developers, the fact of the matter remains that average users can and do get their hands on prerelease software. This leads many non-savvy users running applications not meant for the new release and often leave negative reviews on the App Store because the app is incompatible with the new OS.
For example, see this blog post: http://mbarclay.net/?p=1317
One potential solution to this is to disallow app reviews for users running unreleased OSes. Another is to tighten up the developer program so that each subsequent beta release allows more and more devices to receive installs. With the current 100 devices per account right off the bat, there is widespread abuse of access to prerelease software.
That would only work for on-device reviews, but those users can always leave reviews through iTunes.
Luckily, most of the larger OS updates also required a new version of iTunes, so they could block it for beta releases of iTunes also.
I'd love to be able to filter reviews by age groups (ie. over 25 only)
Give them any path, even a tricky or expensive one, to obtain a beta of a sexy new mobile operating system and they will, expecting it to work as well as the other "betas" they've used online.
As for the bug reports in the review area, I imagine this is for a couple reasons.
First, it's the easy way to give feedback to the developer. I don't know of a way to provide feedback within the App Store architecture other than through the review process. Any other method has a chance of being painful. Being forced to sigh up to some developer site bug tracking system, and then getting sent email after email notifying you of every status update. Of course, I also have to search around for how to contact this particular developer with a bug. There is no easy method that I know of to do this.
Secondly, it's a sure way of getting the attention of the developer. Reviews directly impact the developer, and an email directly to the developer might not get a response as quickly as you want.
I'm not saying either reasoning is correct, or right. I'm not judging the reasoning. I just imagine that's the two biggest reasons for using reviews for bug reports.
Only info which I get is that some users don't understand information on the screen - my bad, I should write it better - but I'm even not able to help this person.
When I get info from user (mainly by special feature which let user to send "bug report" to my by e-mail) about problem I usually answer in 5-10 minutes ;-)
I think the best solution may be to give to developers ability to respond to authors of review, maybe it should be done in this way that user which will set 1 star will be asked if she agrees to get possible response from developer or something like this. Everybody will gain something.
A simple "Support" link showing up a pre-filled Mail controller in our app has proven to be very helpful with that and drastically reduced the bug reports within the AppStore.
Nobody ever said beta testing was easy, and implying that it should be demeans both the testing professional and the developer.
I also think there's a place for non-developers to be running the beta -- if they are responsible citizens. I'm a dev, but not much of one yet. I'm on the beta, and I have discovered plenty of broken apps. When I discover issues, I communicate them directly to the app's developer. As a result I have also ended up beta-testing a number of apps for said developers. This is just as legitimate a use of beta-testing as developers testing their own apps.
Probably not that many, but each developer gets 100 UDID to activate, and I've seen many adding random UDIDs on their accounts for people who wanted to install iOS5
> they would also prevent legitimate reviews at the same time.
There is no legitimate public review from a beta iOS device. Legitimate reviews from iOS-beta testing will be done directly to the developers, not via the AppStore.
However, I do not know if that NDA has clauses about adding random third-party UDIDs (though developer accounts have been banned for that in the past), and in any case it does not bind those getting their UDID added who do not own developer accounts.
I think it's possible to install the beta without a dev account if you know a developer who's willing to provision your device.
But still, there's gotta be quite a few people with developer accounts who really should not have them. Every time a new beta arrives, the Apple developer forums are full of kids whining that their games don't work. (Not that the dev forums are much better at any other time...)
I guess this could be a problem for developers whose plan is to develope their first app for a yet-to-be-released version of iOS, but I'm guessing that is an edge case.
Apple should make it so reviews are only shown for the OS of the user's phone, so if I am on iOS 5 I can see the beta reviews, but most average joes still on iOS 4 would only see those relevant to iOS 4.
This also solves an issue of users still on iOS 3 with an app that may no longer be compatible for iOS 3 clogging up the reviews for they aren't relevant for most current iOS users.
Currently they roll over the reviews ever version of the App the developer releases. That makes sense because bugs get fixed and features are introduced.
You should not see any difference in the different versions of iOS. Though it might perform differently on different hardware.
I feel as if Apple hasn't really done very much to help the developer note what platforms they are targeting in an easy way. If you don't want iPhone 3g users to use the app, they shouldn't be able to download it that that device in the first place etc.
Wrong. Any OS feature available only in version N and above may be used conditionally while the application still works on versions N-1 and under.
Case in point: Apple's Game Center is only available from iOS 4.2 (which excludes 1st generation iPod Touch and original iPhones), and is not available on iPhone 3G.
What if a game developer is OK with a 4.0 baseline, or even a 3.1 baseline so iPhone and Touch 1 users can still play, but he still wants users with compatible devices to have access to Game Center achievements and voice chat?
He conditionally enables the feature.
> If you don't want iPhone 3g users to use the app, they shouldn't be able to download it that that device in the first place etc.
Set UIRequiredDeviceCapabilities correctly and that's done for you. For instance, the 3G iPhone uses ARM v6 and only provides OpenGL ES 1.1. Require armv7 and opengles-2 and you've ensured your application will not be installable on 3G iPhones.
For an example of that, see Infinity Blade [0] which clearly states it is not compatible with the 3G (more precisely, it specifies that it's for 3GS and 4 in the "required configuration" section) and which will not install on a 3G.
[0] http://itunes.apple.com/fr/app/infinity-blade/id387428400?mt...
This is not true. Sure, if you were the ONLY developer with 0 reviews on a new OS version and everyone else had a bunch, it would look bad, but if every app had 0 reviews, you would be no different. And if you had a large user base on one version of the OS you'd likely have many reviews quickly.
> You should not see any difference in the different versions of iOS. Though it might perform differently on different hardware.
Certainly things are often mostly backwards compatible, but there are often changes at the API layer which break some things or add features that are available on the new version that weren't on the old one. For example, Apple added the backgrounding/multitasking mojo with iOS 4 but until an app was updated from iOS 3 and could make use of those APIs, the functionality wasn't enabled. Does it make sense for users of iOS 3 to see reviews of an app, not yet updated for iOS 4, of iOS 4 users bitching about how the app doesn't multitask right when multitasking wasn't enabled in iOS 3?
> I feel as if Apple hasn't really done very much to help the developer note what platforms they are targeting in an easy way. If you don't want iPhone 3g users to use the app, they shouldn't be able to download it that that device in the first place etc.
This is a decent idea, however most things DO work from one OS version to another, so barring people from using an app on their new device seems like it would often be an artificial limitation. Further, it is tough to beta test an OS if there are no apps to run on it. I think this concept of splitting the reviews would solve the problem of clutter in the current review system without barring people from doing the testing that ought to be done.
Further, some kind of hybrid could be done, such as maintaining the star rating across OS versions (so people could have an accurate representation of the general reliability, etc. of a given app) without clogging the review systems of different OS versions with irrelevant data and unwanted complaining about incompatibility with the new beta.
Once they are out of beta the feedback feature can be turned back on.
These users weren't developers, were essentially pirating the software, didn't file bug reports and clearly just thought that it was something cool to get early and didn't understand the implications of it being a beta.
There have been numerous threads on that, starting as soon as iOS5 became available.
> These users weren't developers, were essentially pirating the software, didn't file bug reports and clearly just thought that it was something cool to get early and didn't understand the implications of it being a beta.
And those who pointed out these issues were generally mercilessly down-voted.
Check the OS version against a list of tested versions.
If the version isn't in your list of tested versions refuse to run, or explain that the software may not work properly and please do not post reviews.
They seem to do a decent job with versioning major changes. But our App breaks on iOS5 because of subtle changes.
Point being...there are far too many moving parts to expect them to get everything right in a beta release.
There might be a warning already, but if there isn't Apple should make it very clear that every iOS developer has provided technical support contacts and other helpful links that they can use to report bugs. Bug reports don't belong in the review unless the developer never responded to your contacts or never fixed your bug after contacting them.
I also think that the developer should be able to contest reviews that solely create FUD and aren't really reviews at all.