Note that you can also use Cmd-down_arrow, because Cmd-up_arrow and Cmd-down_arrow can be used to navigate up and down the folder hierarchy, and the latter also works if you have a file selected rather than a folder.
Edit: Cmd-Delete will delete a file. I think the philosophy is that Cmd is used for commands, and Delete by itself is meant for text editing. One thing I like is that Cmd-Shift-Delete will empty the trash. I'm unaware of a keyboard shortcut for emptying the Recycle Bin on Windows, or any way to avoid the confirmation dialog when doing so.