When you say humans "goto space", you're probably talking about going to orbit. As such, the problem isn't that space is "up high", it's that fast is "very fast".

This "XKCD What If?"[0] explains this concept pretty clearly.

[0] https://what-if.xkcd.com/58/