Surely the bank's website offers the same security features, so that accounts may be accessed without a specific phone with a specific app. If not, then that bank should be avoided.

You can still pay in person using chip & PIN or use an ATM without the app. This only applies to online shopping AFAIK.