If you're concerned about the long term effects of the vaccine, you should be double concerned about the long term effects of the much more complex and much less understood virus.
Albeit, I do not believe it is that common, it's still possible. However, if you isolated in a remote location in the woods miles away from humanity, then perhaps you'd be safe, I suppose.