I think the best city for this is Budapest, which in addition to swimming pools it has termal waters. Very nice to go during winter months.
Very underrated place, but I suppose it would have more visitors if it weren't for the language barrier. I remember we couldn't figure out which one is the men's room, because the information was only given via text and that was years before smartphones.