Right clicking on a shape brings up a Delete Shape option.