The most important thing at this stage (and the next stages as well) is to not waste time on things that are irrelevant, so don't make banking more complex than it needs to be. as you grow, your CFO or maybe even COO might have their own processes and preferences, so just keep it as simple as possible till then.
If you are really at the stage that you need a more robust system, brex and stripe are pitching themselves as tech startup friendly for credit card issuance and control, but truly any major bank will do. I've worked closely with Bank of America and Wells Fargo before and my local regional credit union with great success. I was not impressed with US Bank, Keybank, or Chase Bank. I've been really happy with my local credit union though. they have everything and I can speak to high enough ranking people to make sure things keep running smooth. And of course they love nice piles of cash sitting in the accounts.