Support for 100MB APKs on Google Play
android-developers.blogspot.com
android-developers.blogspot.com
Google Play makes you jump through a bunch of hoops to ship an oversized APK, and the tools they provide are totally inadequate. (For example, see this years-old bug affecting expansion files: https://code.google.com/p/android/issues/detail?id=61344)
Raising the basic size limit to 100MB doesn't fix any of the underlying problems. In fact it may make things slightly worse for users, because some developers who would otherwise have been forced to ship variant APKs or an expansion file (more work for the developers, but easier to download for users) will now just ship a single bloated APK instead.
When you get an app update, why do you still have to download the whole thing? Why not a binary delta? For that matter, why can't the developer upload a binary delta to the store? The Play Store holds golden copies of each version of the APK, after all. Instead, we have the clunky expansion file system, and each developer has to do the work.
When shipping variant APKs to different phone sizes / architectures / etc, why does the developer have to build and upload each APK separately? Why can't the Play Store just do the "app slicing" automatically? In most cases it already has all the information it needs.
I realise that all this stuff is hard work, and my complaints really boil down to "Google should do all that hard work for me". But they have thousands of engineers working on Android and Google Play, so I don't think it's too unreasonable.
I like the idea of binary deltas and expansions if handled well, and I imagine a service that makes it easy for developers to distribute in this way would be a goldmine.
iTunes Connect is overall a bit nicer to work with (but often fails with completely incomprehensible errors, usually involving certificates and code signing). The new "app slicing" thing Apple are rolling out sounds very good. On the downside, it's a very closed system. Apple don't want you to script or automate anything, they want you to use their tools, of which some are slick and some are very rough.
Amazon's store is similar to Google Play but much worse overall. Their web app is very clunky. Good: no APK size limit, so no expansion files needed! Bad: big APKs need to be uploaded via SFTP(!??). No clever binary deltas or anything as far as I can tell, just massive uploads and downloads for everyone.
In the Humble Bundle, a lot of the work is done manually. That sounds bad but they've actually been a real pleasure to work with. It's great when you're doing a Humble release of an existing game; do a few code tweaks if necessary, upload to Dropbox and mail them a link + hash. I don't think they do anything fancy in their back-end, though (and they probably don't have enough engineering capacity to do so).
I'd be interested to hear how Steam compares.
Android updates have been incremental updates for three years now (introduced at the 2012 I/O, and immediately enabled). Your point on that is confused in any case -- why can't it do delta updates (it does...), and why can't the developer do it, but also why does the developer have to do it? You're incredibly complaining about a situation you invented.
And larger APKs are a problem because now the ridiculously easy tools that Google already provides (generating thin binaries for varied targets is extraordinarily simple. A bug regarding expansion files...what is the point of even linking that?) will somehow be unused?
Your post is a very poorly informed rant. I have no idea how it sits on the top.
I'm just frustrated that they're just doing tweaks, rather than any deep fixes. Obviously the platform isn't doomed, because the competitors aren't much better!
Android updates have been incremental updates for three years now.
Hmm, OK! I have seen some news articles around that, but I can't find an authoritative bit of documentation or official blog post. If you have a link handy, that'd be great.
I've observed in practice that all my test devices seem to do full downloads rather than deltas, though many of them are running older OS versions. And I'm mostly concerned about expansion files. It's possible that the APK parts use deltas at least some of the time, though again, in my spot checks that doesn't seem to be the case.
why can't the developer do it, but also why does the developer have to do it?
What I'm envisaging is, the developer builds APK version 2 locally. They send a v1 -> v2 patch to the Play Store (which already has the master copy of APK v1). Use checksums to ensure it's all working as expected, and in the worst case fall back to a full upload of v2.
Maybe from Google's point of view it isn't worth spending any engineering effort to help developers who don't have access to really fast uploads. But it's not rocket science and it sure would be nice.
A bug regarding expansion files...what is the point of even linking that?
I think it's relevant because their recommended method of shipping a really big app is to put resource data into an OBB (a virtual filesystem), but the tools for building an OBB don't actually work reliably. Worse still, Android itself can't mount an OBB reliably! So if you care about stability and/or supporting a decent range of OS versions, you can't use OBBs at all. You have to use a zipfile or something as your expansion file, and manage it yourself. Not a huge deal, but it's more work, and it all contributes that little bit more to the bloat many people complain about.
It is absolutely enabled. It is absolutely working. There is no conspiracy about this. It is one of the reasons most updates take hours to propagate.
You complained about managing thin binaries, so I don't see how the developer managing that would improve anything, not to mention that it adds difficulties to cryptographic signing and verification.
OK, that's good news! I stand corrected.
It would still be great for expansion files to get the same treatment, or better yet to abolish expansion files and just allow big APKs.
I'm a little sceptical that absolutely no documentation or feedback is needed. Surely there are things that I as an app developer could be doing to make the deltas smaller.
You complained about managing thin binaries
To clarify, as an Android developer you can manage thin binaries yourself, as a way to reduce the download size; it's just a hassle. My complaint is really that the Play Store should do that app thinning automatically.
Apple sort of presented it like the App Store is picking and choosing, but it's the enhancements to Xcode that actually enable the functionality. Google could add some tooling improvements to make it slightly easier, but really they've already done all of the hard parts.
Expansion files were a hack, and remain a hack. They are necessary for large games obviously, but really 100MB encompasses the enormous bulk of apps with ease. It would be ideal if Google simply abolished expansion files and folded the sizing into normal APKs.
Yes, but given the quality they deliver, I seriously doubt their skills.
Before the downvote rampage starts, just check:
- Bug tracker for each Studio release
- Broken NDK toolchains
- No proper support for shader and Renderscript debugging
- How they managed to have one of the worst JIT and GC implementation than many embedded Java vendors for so many time (don't have an ART enabled device to check how it improves over Dalvik).
- How they aren't able to deliver a bug free Support Library, even though it is visible their own teams (e.g. YouTube) have workarounds for the bugs
- No roadmap on improving the Java situation
- No roadmap for NDK improvements
But hey, just watch all the goodies they deliver with Play Services.
http://www.androidcentral.com/what-are-delta-updates-and-why...
It seems like in practice expansion files never use deltas, and (as far as I can tell) APKs generally don't either. I could be wrong about the APKs, or maybe my updates are always big because I'm using native code.
[Edit to add: my paranoid theory is that they pre-announced this, but then never properly shipped it. But I'd love to be proven wrong!]
[Edit edit: OK, it sounds like APKs already get deltas and it all just works invisibly. But I don't think expansion files get deltas.]
I'm not at all surprised if most developers just ignores the patch option all together and just re-uploads a new main expansion file every time any content has changed.
Besides, while seemingly well documented[0], IME expansion files requires quite a lot of undocumented magic. Including patching Google's own source code. Code that they recommend that you vendor into the app rather than include as a library.
Further the user experience is less than stellar. On many devices the expansion is only downloaded after the app is started the first time. So the app needs to work even while the file is missing, and at that time manually request that the system retrieves the file.
BTW, did anyone notice the fine print of this announcement? Only devices with Android 4.0 or newer supports 100 MB apk:s. If your app supports 2.3 - i.e. some 70 million Android users - you're stuck at 50 MB and will have to keep using expansion files to serve those users.
[0] https://developer.android.com/google/play/expansion-files.ht...
I'm glad they included and expanded on this point. I used to be stuck on a gingerbread device that wouldn't let me install anything larger than a megabyte due to storage issues. I was always grateful to devs who kept their APK sizes in the kilobytes. Now I have a Nexus 5 and install 30MB+ APK's without hesitation because that seems to be the status quo. Does anyone know why most apps these days are so bloated in size? Is it all the frameworks and libraries we use?
It's one part. The other is graphical assets: You have to provide them in half a dozen DPI versions (vector graphics are still hit or miss), with nowadays ridiculous densities (QHD on 5"? Oh come on). That's a lot of essentially wasted data.
That's probably the understatement of the day. The Play developer console is such a pain in the ass to work with that I'd rather not.
However, a build script to handle it all for you during upload would be a great way to solve it.
The API for this seems to lack the features that you would need though [1]. It doesn't appear to let you upload apps for only certain devices - unless you would specify it in the manifest of the app somehow?
Also amusing - the API says "Do not supply a request body with this method", yet obviously you'd have to supply a request body - the app itself! I have a feeling this is not a well-used API.
[1] https://developers.google.com/android-publisher/api-ref/edit...
Or Google just doesn't give a fuck. There's various bugs with uploading expansion files that are documented from day one and still unfixed.
I imagine you could now write some gradle code to do most of it automatically...
It seems like apps written in Java have less of an excuse -- no fat binary needed and the system APIs are much richer. But maybe people tend to use a lot of frameworks there too.
Also, graphics can take up a surprising amount of space in normal apps, not just games, if you ship variants for all the different screen sizes. On iOS, for example, you include a startup image for each screen size / orientation. That could be a dozen or so full-screen images! And screens are massive these days.
Modern "flat" UI images ought to compress much better, though...
Overall, I don't think it's just bloat, there is some justification. But I bet most apps could fairly easily be made significantly smaller.
It's not as if one of the major use-cases is to copy APKs between devices with different architectures and have them work easily, and I'm pretty sure it should be possible to implement something that detects which one the device is before downloading, so how did this situation occur?
On iOS, for example, you include a startup image for each screen size / orientation. That could be a dozen or so full-screen images!
Unless the images are truly very different (and I don't see why they would be), isn't it possible to just include the highest resolution images and resize if necessary?
Modern "flat" UI images ought to compress much better, though...
If anything, "flat" UI means there shouldn't be a need to store much in the way of imagery - they can be procedurally generated very compactly in the code itself, or stored in some sort of vector format (even better) - but then again, I've seen people use huge solid colour bitmap images for UI elements in other (desktop) apps.
Libraries/frameworks are slightly more tricky but once again, there already is a solution: identify them as dependencies by something like a hash, and don't download if they already exist. You may end up with multiple versions of the same "module", but at least you won't get multiple copies of the exact same one.
I'm really astounded that such wasteful practices are prevalant in the mobile device world, where bandwidth and storage are far more constrained than for desktops. It's almost as if they deliberately chose the most bloated way to do everything...
If you only have 2 architectures, and .so files in tens of kilobytes, it's just more convenient than building and uploading multiple APKs. When you have 3+ architectures, and 7MB javascript runtimes for each, it indeed feels ridiculous. Looking at you Titanium, NativeScript and now React Native.
> Unless the images are truly very different (and I don't see why they would be), isn't it possible to just include the highest resolution images and resize if necessary?
Good icons can be quite different in different sizes: in smaller sizes you remove details, adjust line thickness etc.
Another, now historic reason is that early Android devices were underpowered and loading correct size graphic vs loading-and-resizing was important for performance.
Doing the work to split packages up into different architectures helps a lot (if you use a bunch of native code), but then you have to deal with the fallout of people downloading from google play for their device, and then posting the slim binary on the web that other people with different devices are going to download and install to incompatible phones.
1. APKs will expand when installed (dalvik-cache directory)
2. APKs need to store user data and cache data (think Google Play Services, Chrome)
3. Google's “ecosystem” takes up more than 1.5GB from user-space (services, chrome, maps, google app, gmail, docs, slides, sheets, photos, drive, hangouts, street view, tts-engine, youtube, music, calendar, games, webview, play store etc)
4. User will want to store a few photos and videos on his phone and maybe have some offline music.
[1]: https://developer.android.com/google/play/expansion-files.ht...