For me, the UI performance is the single biggest reason why I don't use Android as my primary phone. No matter how fast the underlying action is, the jumpiness and occasional lagginess makes it feel slow to me.
It frustrates me - there's so much I like about the OS that so many good things can be erased by leaving the visible perception of being slow.