Literally does mean figuratively, since a few centuries ago, so no need to change the meaning of the word. Regardless, people understood what was meant, it seems only you are taking it, well, literally.