It’s for many, many things besides letters, and was designed to be so from the beginning. Pictograms of various types have always been in Unicode, and even in many predecessor codesets.
Unicode is for text symbols, some of which represent one word (e.g. &, U+2116) and some of which represent multiple words (e.g. U+FDFD). Other non-letter characters include numbers, punctuation, and so on.