This is probably the story in page 220 of [1], beginning at "Von Neumann got into trouble at the end of his life because he was really a frog but everyone expected him to fly like a bird.".

[1] http://www.uvm.edu/pdodds/files/papers/others/2009/dyson2009...