It's really irritating for your users. They might not consciously be able to point it out as the cause of their irritation, but they'll like your app a lot less.