On servers, in almost every situation, I'd rather have the server reboot than have some random process crash and leave the system in a potentially broken state.
On desktops, in every situation, I'd rather have X, my window manager and an emergency terminal pinned to RAM so I can always decide what to kill for myself. (https://github.com/stiletto/angrymlocker helps with setting this up.)