You can delete files immediately (i.e., skip moving to the trash) in Finder by holding the Option key in addition to Command + Delete (or Backspace).