I eventually found David Galvin's calculus notes[1] from University of Notre Dame. He basically follows Spivak closely, but reorganized the material a bit in response to user testing. The notes aren't perfect, but much much easier to follow. Same experience with Terence Tao's linear algebra notes[2].
I think book authors, even very highly respect ones, often kind of suck because they optimize for writing a beautiful book, not for minimizing student confusion. Once you struggle through the confusing parts, yes, the book is beautiful. But it's supposed to be written for people to learn, not for experts to appreciate! Notes written by professors who teach smart kids, optimize for minimizing confusion, and do real user testing are often much better than the best books, in my experience.
[1] https://www3.nd.edu/~andyp/teaching/2020FallMath10850/Galvin...
[2] https://terrytao.wordpress.com/wp-content/uploads/2016/12/li...