For some systems, it's just a matter of logging in to the server and running a single command line, like "sudo yum update bash" (replace "yum" with apt-get, or another package manager).
You can leave it, but know that you're leaving your clients vulnerable to things such as:
- stolen data - data loss - compromised/corrupted/deleted backups of data, code - site disruption - botnet participation - illegal file dump/trading space - unremovable rootkits
Having said that, I believe you should be safe if you don't use CGI to run your apps.
The earlier you plug the holes, the better.