Ease of use really matters. If you think of your product workflow being a series of micro solutions, then user delight implies you picked the right micro problems to build a solution for. For instagram, a key micro problem was it took too long to upload photos from a phone. They solved it by pre-uploading it. A great talk in this regard is 'Stanford to startup' by the instagram founder.
Before stripe came into the scene, I tried to do a paypal integration into an app once. It was an absolute nightmare. The one time I did stripe integration, my grandmother could have done it :)
Also, it doesn't matter that every bank builds an API once Stripe becomes (or is perceived as) a market leader. People view it as genuine and provide disproportionate value to the leader. People will go out of their way to use the market leading product even if they have to pay more. When you're in a store, think of how many times you buy the more expensive product because you see that as the default brand in that space.
But wasn't PayPal the market leader? Then how could Stripe have taken over? Well, PayPal sucked (for developers). Stripe was 10X better and took over while PayPal slept.
To replace the incumbent you need to be 5-10x better, otherwise nobody bothers to move over. The key to that is to pick the right set of micro problems to solve.