The downside is they are encouraging you to share passwords, as you say, which isn't driving the right customer behaviour.
More critically; in about 18 months the PSD2 Secure Customer Authentication guidance comes into force and this sort of approach (sharing credentials, which everyone basically refers to as "screen scraping" in its various forms) will be dis-favourer, to the extent that banks might have to go to great lengths to try and stop it. Teller might have to go forward fighting continual reverse engineering battles.