Command delete works here. It makes some sense as it's not likely you'll accidentally delete one of your photos that way.
> Or, when I close all the windows for a given program (usually there is just one), the program stays open until I navigate through the menu to explicitly quit.
That's not necessarily true in Lion (through a feature called Automatic Termination which requires the app to explicitly support it). Take Preview again for example. Open a photo, close the photo and then go back to another app. Check the Dock. This actually drives me nuts as I commonly switch app context's with command-tab and having a disappearing app makes this difficult. It seems like an odd feature to me because the process actually remains alive (check Activity Monitor), it's just in a state that makes it easy to kill if resources become scarce. I have 20GB of RAM, resources very rarely become scarce so all it does is end up hiding stuff I'd rather see.