Countries with unreliable power grids usually don't have drinkable tap water for that reason.
It can also be because the water isn't treated (ie. it is just rainwater in the pipes). Water treatment actually isn't very expensive though - collecting the water and distributing the water are by far the biggest costs.
How does this happen? Are they not sealed?
Using safe potable water to flush toilets is silly. Splitting drinking from other uses is far more efficient.
This isn't universally true. Pipework (laying+maintaining), collection and storage is generally the expensive bit of any water system. The actual purification is fairly cheap.
That means having two sets of pipes to every house (one for drinking water, one for mid-grade water) usually costs more than just one set of pipes and having all water drinkable. Thats why it's rarely done.
The alternative is you use the pipes for non-potable water, and tell people to buy bottled water for drinking - more than half the world is in this position I believe. It helps that people drink perhaps only 1% of the water they use - so bringing 1% of the water via bottles on the back of a motorbike is viable.
Could you not recycle clean water? e.g. a rough filter & chlorination of shower/dishwasher water to use for other things.
Some stages of a dishwasher without harsh chemicals might be used to water a lawn?