The local 50 meter pool here is always filled with people young and old (and on no way elite) that just swim. You don’t need to be very fit to be able to swim 50 meters without a break.
It‘s a different kind of swimming compared to just hanging around and having fun, sure, but it always seems to be quite in demand when I see pools like that.
(The swimming complex close to me has two Olympic size swimming pools and one of those is open to the public, the other is used by actual elite athletes but also kids who are learning to swim. There is also a small 12.5 meter pool with adjustable depth to learn swimming and for aqua fitness courses.)