The motherboard has an "up" direction if you lay it out flat.

Eww that's weird, ok in your case you are out of luck. That's pretty strange though, I have a ton of USB-A devices here and they are all in line with the way I outlined. So that's a bit of a surprise but it may well be an exception.

As for the ones that are sideways: the 'bottom' of the case is the bottom of the motherboard, so if you think of it that way it might still work for all but the top side ones.

We live in a right handed universe so even sideways has a correct answer.