Concerning, but still much better than giving a third party your username and password which could be used to initiate a transfer or something.

That really depends on your threat model. If someone already has access to your email account it's game over anyway most of the time. I don't see much of a problem there if you are not worried about your ISP / mail provider taking your data.

So long as the bank's email server connects directly to your email server over an encrypted connection it's just as secure as anything else.