Well I was running it on a 2GB VPS, I have since November switched to a 4GB VPS, no issues since then but it's still early days. I haven't benchmarked either VPS, the 2GB I had a single dedicated CPU core, some 3.5+ghz xenon. I didn't check what my cpu allotment was on the new VPS, it was a $65 a year black friday thing. Moderation is not an issue, I only have 5 users other than myself.
Here is the last time I complained about matrix. https://news.ycombinator.com/item?id=25100873
I saw your comment and considered posting a facetious comment about how you would need 80TB of ram and at least twice as much disk space.... but that would add nothing to the discussion.