Yeah, that's how the banking system works. Every single check in your checkbook has your account number and routing number on it, and those two numbers are literally the only pieces of information necessary to transfer money out of your account.

What the system lacks in technical security is (supposedly) made up for by legal protections/processes--yeah, it's incredibly easy to take money out of an account, but that transaction _will_ get reversed if it was fraudulent.