Colo is most performance per buck. Cloud is least performance per buck but no headaches.

There is also the option of managed metal. You get raid, monitoring and super performance.

I used to think VPS was the middle ground but now I like managed metal best.