Not quite - if someone steals your 1Password (or equivalent) database with TOTP seeds in, they need to crack the password on that, then have full access to everything. If they get the password from the other end (e.g. the site you log into), they can probably log into that specific site (they will have the TOTP seed), but not anything else. In general, there are more attackers looking at the site end than at the client password DB end.

Probably wouldn't be a bad idea to have a distinct password vault for TOTP seeds, although they'll be stored on whatever device you generate codes from anyway (hopefully in a secure way). At the very least, it might be helpful for moving between TOTP devices without needing to do the random steps required by different services!