Random example: http://comments.gmane.org/gmane.comp.handhelds.phonegap/3631...
I work for a very popular cross platform mobile application (ios, blackberry, android), and we have dozens and dozens of test devices, a qa department just dedicated to finding differences between devices, and let me tell you that most of the time, special tweaks are required for android that are not required for ios. Surely this is hugely due to the fact that there is less variety in the ios world, but you simply wont find that "manufacturer xyz decided to implement this differently so now the developer must account for that".
Finally, in your analogy of adapting from Android->Android, is similar to adapting from say ios5->ios6. Having been through this several times now, I can tell you with confidence that the initial time of adapting is very similar with equally skilled developers, but the delta of "code complete" to "ready to ship" is much larger on android, due to all of the testing, double-checking, and edge cases. So in summary I would say that you do "adapt a little" with ios, but you end up adapting a lot more than a little for android, if you want to support more than a couple phones, in my opinion.