The fragmentation comes from the woeful OS update situation (API fragmentation) and there being a ton of different manufacturers each producing something slightly different (hardware fragmentation).
Older iPhones don't support all the features a 5 does (panorama, Siri, etc), but they support the latest APIs. Want to use a ICS TextureView for fast GL rendering to a view? 70% of the userbase won't have support.
Writing an image processing app that depends on the GL_OES_texture_float extension on the GPU? Phones from with GPU X will have the extension, phones with GPU Y won't. Need a gyroscope for some fancy UI widget? What about NEON intrinsics? What about an app that relies on all three? Having so many more combinations of hardware is a blessing and a curse, and testing against all the possibilities is a real hassle. Every platform has hardware fragmentation - Android just has it the worst.