One thing sort of hinted at but not explicitly mentioned is that the field you speak in makes a significant difference, especially at the lower end (really famous people will make out well in any case). If you can get onto the corporate speaking circuit, doing tutorials and lectures at corporate retreats, for example, there is a lot of that work, and it pays fairly well. This usually requires a sort of "pop" or "business" angle on your speaking. Even a very engaging lecture about advanced mathematics is not going to get you invited to most corporate retreats, but a lightly mathematically tinged anecdote, or a more business-focused lecture about using statistics to analyze markets, might do so.

I've noticed in game-design, for example, that over the past 1-2 years by far the best way to go if you want to make money speaking is to get into "gamification", talking about how companies can use game mechanics and badges for engagement and that kind of thing. Even if you're a better speaker when talking about actual game-design, the markets are just very differently sized, so it's more lucrative to give an "ok" talk about gamification (lots of clients) than a great talk about game-design (many fewer clients, unless you're Will Wright level).