Wow, can't get it to go away on Firefox 33.0 Ubuntu. I was able to clear it by manually deleting the info from the permissions.sqlite3 database as described by agwa.

Very clever!