In 15+ years of using OS X (starting with 10.1.5), I’ve never bothered. Why make more work for oneself?
The only trouble I’ve ever had with an upgrade at all was a recent minor version (10.13.2 to 10.13.3), which almost nobody would bother with a fresh install for.
YMMV, of course.
I do sometimes wish it were easier to start fresh with Homebrew. I guess that would be a reason to wipe and restore.