For non-Germans - the Elster is the magpie. According to Wikipedia, in German folklore the magpie is a thief.
As a Brit I do not associate the magpie with stealing per se but rather with having a fascination with shiny objects like jewellery - which also sort-of works, for the joke.