I have been plugging and unplugging multiple displays into my laptop since March. Usually I have emacs running locally (xwayland) and an emacsclient over X11 to a remote server. I have not experienced any issues so far. Maybe the readme is too conservative in this regard?