Well, to each their own. I never have to restart Emacs. The only time I restart SLIME is when I do something I shouldn't, like redefine a struct or something. I'm sure I could avoid restarting SLIME, too---I just never bothered to learn if there is a way.
BTW, all software that changes will have a bug from time to time, but Emacs and things related to Common Lisp are like at the very bottom of the trouble-frequency charts. I can't think of any software that I use regularly that gives me fewer problems. For the record I never encountered a bug in either (which doesn't mean they don't exist).