I gave up caffeine more than a year ago, because I didn't like the idea of being addicted to a drug. I have a cup of tea once every couple weeks or so, but that's about it.
The only improvement I've noticed is that I don't suffer withdrawal symptoms if I don't have caffeine. If anything, I think my mind feels a bit slower without it. I think I might take it up again.