You have it the other way around, security starts with privacy at its absolute minimum. If data is sent to a third party every time you tap something on the phone, you are using an insecure phone, regardless of what complex hardware they are using.
> But using a custom ROM doesn't necessarily mean you have to permanently unlock your bootloader, so that argument doesn't make sense.
True, depends on the phone though, some of them cannot be locked again and there's no way to completely fix those phones with a better ROM.