-Galaxies are not accelerating, space is expanding.
> Imagine there are some objects in space all moving in random directions and speeds, relative to Earth...
-No, In your scenario then end result would be a most static average distance between all objects in the universe. As an infinite number of objects come from infinite distances, there would ALWAYS be objects in the neighborhood.
I think what you're imagining is a bunch of objects in a box, give them random vector and then remove the box. If they maintain course, all will eventually move outside the original box boundaries, and away from each other. (not the way the universe is).
They know space is expanding. The primary mechanism we know this is the speed with with we measure an object (moving away) is redshifted. Objects at the same distance from Earth, but opposite regions of space are moving at the ~SAME measured velocity.
There simply is no existing theory which can account for what we are seeing besides space expanding. I'm not big on thinking we understand it all, but in this particular measurement, there is basically zero doubt. Space is expanding, which has the affect of accelerating all objects in the universe away from you, with an acceleration relative to the distance. The more space between you, the more opportunity to expand.