A nice thing with Slicehost is you can start with the smallest, and then if it seems you need more RAM, can restart the same image on a bigger slice.
You will know if you need more RAM by watching your busy server with 'top' over time, and among other things, ensuring that it never begins swapping.
Almost any app can benefit from more RAM, especially if you use various levels of caching effectively (from the automatic linux disk cache through framework-level fragment- or page- caching to HTTP-server/proxy caching).
So when you're sure things work OK, but then want to make them snappy, throwing RAM at the problem (with a little engineering to use it wisely) is usually a good idea.