Any chance that, by limiting RAM usage, you forced your application to heavily swap, clogging the disk and making your machine slow?
I have run a public gateway on 2GB of RAM. Later 4GB because it was subject to very heavy abuse, but it was perfectly possible. Perhaps it is a matter of knowing how to configure things and how to not inflict self-pain with wrong settings.