First heard of this from Frank Ryans book Virolution. It explains a lot of what had been termed junk DNA previously.
Bacteria's pay dearly for each extra bit of DNA they carry, since the energy cost of DNA replication is linear with respect to the length.
Bacteria's with less DNA can replicate more quickly and thus outcompete other bacterias.
The fact that after so many generations they still carry that "junk" which they pay for suggests it really is useful in one way or another.