That's a beautifully typeset book.
Any suggestions on what authoring tool(s) to use to write a book like this that has lots of matrices, diagrams, and notation?
Any suggestions on what authoring tool(s) to use to write a book like this that has lots of matrices, diagrams, and notation?
As for your question, usual answer would be LaTeX + (maybe) some specific plugins (like TikZ for diagrams, or tufte-latex for specific layout, or whatever), but due to some irregularities here I suspect that this particular book is actually hand-made in something like Microsoft Office. I well might be wrong, I'm not sure, it just seems like that.