I learned from this thread that you can actually disable all that. I did so and my spotlight searching sped up 10-fold and now I only get app results. So much better.