Have you seen Thermae Romae Novae? It's a manga with a Netflix anime adaptation about onsen. Very well-made and it was what got me first interested in onsen!
@strobe999 linked to the Wiki article for the manga. The story follows Lucius, an architect in ancient Rome, who timeslips into modern day Japan. He carries ideas from modern Japan's bathing culture back and builds the public baths of Rome. It's a comedy, but a very informative one at that.