To go through your points:
* If the "wrong" GPG binary is loaded, the game is likely already up: If an attacker has that degree of control over your machine, they're already on the other side of this airtight hatchway. They can attach to your running programs, read memory contents, and so forth, so compromising the GPG binary isn't necessary.
* Keys are handled by GPG, which normally should mlock key material.
* Part of my plan to switch to handling pages (rather than allowing data to be managed via a std::string) was to be able to use mlock() properly. I'll be adding that soon.
* Since the GPG binary is running as you, it could do Bad Things to the underlying filesystem (or filesystem exported by FUSE), but you do raise a point about closing file descriptors for general cleanliness.