downsides is that wood is porous and full of things that dont do well in extreme cold and vacuum. These wont last long, will become very brittle, and have the potential of offgassing things that hardware doesnt like.
My first assumption when thinking about wood is the one that you are having. But my second assumption would be that they've probably thought about the same things.
Conditions in space are extreme but at least they are stable and known so i'd bet we would know how to treat the wood for this environment.