It's not the growth during washing I'm worried about, but that not rinsing them will leave them covered in a growth medium that will significantly accelerate their growth over the next hours or days.
But yeah at times I have the same thing running on my head.
I was able to reduce my water bills by almost 30% just by manually cleaning the utensils and reducing the washing machine usage.