Intergalactic travel may be implausible, but I've seen results that a civilization using Orion-drive-level technology - that is, using the kind of thing we could build today if we had the will to do so - could colonize the galaxy in a mere hundred million years. There's no scientific or technological barrier to that, only a political one.