Notifications don't appear to be E2EEed. They are encrypted in flight. But they do change formats while on the servers. For example, the application server contacts the push server using Web Push (like a webhook), but the communication from push server to the distributor app on the end device is using some other protocol like XMPP, Server send events or WebSockets. This obviously requires decryption.

But the encryption shouldn't be a big issue as @morith pointed out. The application server can send a notification to the end user application simply asking it to refresh. The second image on their home page (https://unifiedpush.org/) illustrates this.

IIUC notification contents are application-defined. So if you're building a weather app and don't care about encryption, your content can literally be "It's going to be sunny." Signal, on the other hand, would only send "please refresh" and then it's the app's responsibility to act on it.