Sure, you have to have some trust. Even if you had access to the source, you wouldn’t know that the OS on the phone does what it claims. Then even if you audit and compile the OS from scratch, you also need to audit and compile from scratch the compiler. This is a long process, because you need to bootstrap the compiler compiling somehow. After all that work, you still can’t be sure. Because you also need to build your own semiconductors. You don’t really know that your CPU is doing what it claims, or that there isn’t a secret radio chip embedded into your RAM.
You need to trust a lot of people to have any sanity. Adding WhatsApp on top of a pile of hundreds of companies isn’t that extreme.