In Linux, just run an app as separate user. Linux is a multi-tenant OS, so users are well protected from each other. If you need to share files with program, share them via a shared folder, e.g. `/tmp`.
In theory, the X11 Security Extension would seem to provide a middle ground. On the plus side, I don't notice any performance impact when running Firefox as an untrusted client. However, most programs aren't coded correctly to coexist with it. For example, Firefox crashes regularly when running as such (via SIGSEGV no less, which is its own yikes). Not only that but many programs that are themselves trusted (i.e. normal/default X11 clients) will misbehave if they are simply near an untrusted client: LibreOffice Calc, for example, will lock up hard if the untrusted clipboard is in use.
It just doesn't work in practice.