IMHO, the number of students that would benefit from learning about AST transformation and code gen vastly outnumbers the folks that would benefit from learning about finite automata transformation (and I took a dedicated course in that).
There simply aren’t that many folks working on regex engines.
Another reason, in the context of compilers, is that there’s not much to be gained by micro-optimizing lexing and parsing. After all, a code is compiled just once and executed multiple times. So you know where much of the fun lies. The undergrads unfortunately usually miss out on the meatier part of the compiler course.
You'd be surprised: there's even a dedicated annual international conference on implementation of automata that has been going since 1996: <https://link.springer.com/conference/wia>
There are constantly new algorithms, libraries, tools and applications around regular languages and regular relations (an extension of regular expression that is more elegant, symmetric, expressive and takes a lot of the ugliness of regexes out). Simplifying somewhat, finite state transducers are NFAs "with output".
See e.g. FOMA at <https://fomafst.github.io/>.
When I took a compilers course that used this book, the professor had his own syllabus and would use the dragon book in support of his plan. We jumped around quite a bit and certainly didn't use all of the book.
That course was one of my favorites and had a huge impact on me. I haven't written any compilers, but have used a lot of the concepts (especially state machines) over the years.
Yes that's true but there are A LOT of leetcode/hankerrank problems that are trivially solvable when one has a solid grasp of DFA and state machines in general!