In my mind, if you're running potentially-compromised code on your machine, you're already hosed and the display server isn't going to protect you.
If you are doing X11 forwarding via SSH it defaults to a more restricted configuration that only allows a more restricted access to the server (no direct sniffing of the input devices for instance).
Do you expect that random webpages are ableto steal all data from all applications you run, and all files you have? Including passwords etc.