Last April, I did the credit card tap in NYC. Last month I was there again and tapped my iPhone and never had an issue. It felt just as fast or even faster since my phone is usually more accessible than a given card in my wallet.
I wonder if the Tube’s tap in tap out setup contributes to the issues you see.