Michael Collins. One of the loneliest humans in history. Apparently some other command module astronaut was a bit farther away from any other human than Collins, but I can't find a good source for that.
He wrote an excellent book Carrying The Fire that's widely considered to be the best written by any astronaut. It gives a broader overview than just the Apollo missions and covers the Gemini and Mercury programs that prepared for the moon shot.