Deleting is definitely the feature I'd most like to have.
I don't generally love the idea of taking over right click behavior, but I've pushed out a patch to allow removing elements using right click. This is an admittedly poor solution (especially for lines, as they are very small click targets). Hopefully I'll have a chance to develop a more robust way to delete elements such as the box method that you described.