Hacker News Reader

Comment by masklinn | Hacker News Reader

> In my opinion, the fact Apple has posted any code for iOS 4.3 at this point is a big step in the right direction: they're not perfect yet, but at least they've got 8/10 of the projects up.

The truly weird part, to me, is that with as much (L)GPL code Apple releases they don't seem to have an automatic, systematic system in place to handle that. Even for a project as big as iOS.

View on HN

Apple seems to have a lot of odd processes with respect to code and development tools. One super annoying one is having to re-download the entier Xcode bundle (4GB+) every time they make a minor change.

Yeah, Apple is chronically unable to ship diff updates (or more generally updates). They just can't do it apparently. iTunes update? Download everything. Xcode update? Download everything. AppStore app update? Download everthing. iOS firmware update? Download everything.

All their other updates done through Software Update use a very sophisticated binary diff setup using Colin Percival's bsdiff — the updater checks the current state of your filesystem, hashing the files covered by the update manifests, and downloads the smallest possible update.

The only downside is that the "Checking for new software…" dialog takes fucking forever.

I'd love if that dialogue would give more feedback about what it's doing. I've always just assumed that there's an issue with my connection or their update servers.

Complete, non-diff updates have predictability on their side; if something was corrupted on the user's end, the update will overwrite it, whereas a diff will leave it in an unpredictable state.

I suspect Apple prefers the simplicity and explainability of "here's the update, same thing for everyone" to "well if this smaller update didn't work in any of a huge variety of ways, you could optionally try this other update"

This can easily be automated. You could send a hash of the binary and have a custom patch either pulled from cache or created (the largest "patch" being the whole binary). You'd end up with a fairly complete cache very fast though. Binary diffing is a mostly solved problem. This assumes that there's a list of hashes that correlate to known historical binaries. If the hash doesn't match you fall back to sending the whole thing.

we have a way to detect if it's broken.

Do it by checksum. If the checksums don't match, then redownload the changed parts. This is also fully predictable, but it's also far faster.

> "well if this smaller update didn't work in any of a huge variety of ways, you could optionally try this other update"

Not once has Chrome botched itself during an incremental upgrade of itself.

It's a trade-off between the size of a download and having to prepare, test against and support multiple configurations.

You sound like the only person who's dealt with this problem before :-)

I remember the Visual Studio setup team and their stories about the woes of trying to make downloads as small as possible. You're fighting with:

- wanting the file download to be one step

- what version (or sets of versions?) can you upgrade from? Does this include the one-off patches you were mandated to provide to top-tier (guaranteed bug turnaround time) Fortune 100 partners?

- how do you handle corrupted files or munged bits of the installation that were unchanged between versions?

The list goes on. You'd think things like Setup are boring, but trying to make the experience smooth and good requires and incredible amount of amazingly unglamorous work. It's one of those pieces of the product that nobody notices until it is slow/buggy/has picture of people who are way too happy to be using your product flash during installation.

I think the problem is that doing so is a zero-revenue effort. Why spend man-hours putting together an automated workflow for source release when you can just do it manually (and lazily) every time someone complains? That doesn't justify it, of course, but looking at it that way makes it seem not-weird.

This is risky, because lawsuits can have damage awards. If you decide not to comply with a contract you've agreed to because "it's a zero-revenue effort", that could look bad in court.

Legality aside, it looks pretty bad from a human interaction perspective: "we took thousands of man-hours worth of code from the community, but we were too lazy to spend even one man hour to upload a tarball to our website and follow that community's wishes". The fact that they are legally-obligated to not act like that just makes it even worse.

But is it really an hour? I doubt that Apple uploads the code they actually used: it's likely that code, and then lots of removed components that would reveal too much about how something works (e.g. there are no build scripts or Xcode projects for JavaScriptCore releases, only the code itself).

At least, they would need to make sure that none of their proprietary code slips out. That likely involves more than "svn checkout; tar; ftp", maybe even lawyers to check it.

there are no build scripts or Xcode projects for JavaScriptCore releases, only the code itself

Not true. See JavaScriptCore.xcodeproj in ...

http://www.opensource.apple.com/source/JavaScriptCore/JavaSc... http://svn.webkit.org/repository/webkit/trunk/Source/JavaScr...

... and if Xcode isn't your thing, just run the script "build-webkit" ...

http://svn.webkit.org/repository/webkit/trunk/Tools/Scripts/

... build instructions in plain English here ...

http://www.webkit.org/building/build.html

That's true for standard WebKit, and even the Mac OS X version, but not for iOS.

For example, the latest iOS JavaScriptCore release (4.2, http://www.opensource.apple.com/source/JavaScriptCore/JavaSc...) has no Xcode files like the one you linked does.

I haven't read the LGPL lately, but is that a violation? Do they have to provide enough information to build your own copy?

Yes, it is.

"For a library, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the library."

It has Makefiles. Assuming those build a binary, why would you think they are hiding Xcode project files?

They don't. Well, they might, but not an iOS binary. There's Makefiles for Android and (I believe) the desktop, but nothing setup for use with the iPhone SDK.

In addition to what openbear pointed out, it should be noted that the GPL requires build scripts to be included with the source. (What good would source do anybody without the three-bajillion-line Makefile that builds it?) GPLv3 also extends this to stuff needed to run the code on the hardware it's intended for (like tools to sign binaries so they'll run on locked-down embedded platforms like phones or DVRs).

There's a practical aspect to the GPL, here. The spirit of the license is, "if I can't view, modify, build, use, and redistribute the software, it's a violation."

components that would reveal too much about how something works

It is a significant violation of the spirit of open source to try to keep others from understanding how things work. Mobile Safari may be a competitive advantage for Apple, but the advantages are in the UI, not JavaScriptCore. Google already has a pretty decent JS engine of its own, after all.

For sure, I'm not defending them. I'm just trying to drive home the point that Apple is going to do what's good for Apple. Unless the community makes the promptness of source releases a business concern for Apple, they have no incentive to improve their process. Contracts are only binding insofar as they are enforced, in this case either by litigious means or by negative PR that impacts Apple's bottom line.

> Why spend man-hours putting together an automated workflow for source release when you can just do it manually (and lazily) every time someone complains?

Because doing it manually every time ends up costing more man-hours than having an engineer or two bang up some check/upload script in a morning.

It's not that hard, I don't think.

E.g.,

All GPL repositories live in this /gpl/ directory; it gets tarballed and gzipped and dumped into the /other/ location immediately prior to creating the build image.

Maybe Apple's got some whacked-out approach to code management brought on by some funky OS Classic approach, but I doubt it, they are smart people over there.

Reply on news.ycombinator.com