A serious concern I have about self publishing (which doesn't appear to be addressed, hopefully I didn't miss it) is bit rot and long term format accessibility. arXiv (and traditional journals more generally) provides a reliable, versioned central repository. The contents appear in a predictable format and (mostly) adhere to an accepted set of styling conventions.
Today, you can retrieve scanned PDFs of many (most?) papers from the early 1900s. Other than the lack of color photographs and more simplistic figures, they're largely the same as their more modern counterparts. (But not quite - the lack of DOIs is downright painful.) I suspect that the entirety of arXiv will still be readily available in some form 100 years from now provided society doesn't suffer a total collapse; the same can't be said of a one off website as described in the article. (Perhaps versioning it on GitHub will prove to be enough?)
Related, probably the only frustration I had will Distill was that articles had a tendency to rely on assets hosted by third parties. Those are particularly prone to disappearing unexpectedly in my experience. I fear that one off websites will make this issue significantly more common.