Actually, that's a pretty clever optimization for caching, IMO. The user tends to leave apps in memory, leading to much snappier performance on machines with sufficient RAM.
EDIT: And if you're savvy, you can always manage memory manually and quit from the top menu or from the Dock.