> Does that explain why it's so wobbly when going sideways

Yes. Check out the smooth movement of a vehicle with properly oriented wheels:

https://news.ycombinator.com/item?id=16994136