I have always been concerned about that as well. Although aren't you somewhat safe depending on if you only open it with the corresponding pdf software? I thought the "only" vulnerability was the software it's used with and if you double click it or not.