The Universe is a lot bigger than our galaxy though. Why is it more special to live near the beginning of humans populating the galaxy than, say, living near the beginning of humans populating that new subdivision they're developing out at the edge of my town?
Because the galaxy is a lot bigger than that subdivision, too.
Populating the galaxy isn't that hard? Care to explain?
If you had todays technology and 100 million years to do it it wouldn't be particularly hard. The hard part is getting things done within our lifetime, but that doesn't matter for something like populating a galaxy. Slow ships are just fine, humans can reproduce on the ship. High failure rate and those ships being huge and expensive isn't a problem if you have thousands of years of production to make them.
Probably a lot less time than that if we decided that is what we wanted to do.