That's a common misconception. Plane rental is charged by the time the magnetos are energized which in turn is measured by the Hobbs meter. Having said that, many places often have a minimum number of hours per day to prevent someone taking a plane all weekend for two billable hours.
Flying as a hobby is a lot like boating - the happiest day of your life is when you sell your boat/plane. The second happiest day is when you buy it. It can be woefully expensive because much like a house it's easy to do just one more thing / buy just one more gadget and think well if I just spent a little bit more I could have Y instead of X.
I wouldn't discourage you against taking a discovery flight but I'm saying this in the interest of full disclosure. Back when I learned to fly there was a running joke about the $100 hamburger because unless you had specific goals for your license that's all it was good for - a quick jaunt to the next town over to keep current and have some grub. Considering rental prices these days I guess that's more like a $300 hamburger.
If you learn to fly, you'll pay a rate based on the type of airplane and whether aviation gas is built into the price. After a very quick Google looks like in California prices start at $130+/hr. You need 35-40 hours* just to get the first of your licenses which will let you fly a small plane in good weather. Realistically once you have a ticket any place renting a plane will want you teach you how to fly the model they're renting you, so tack on a few hours more to get signed off in the plane you want.
Once you rack up enough club membership dues or hours in a rented plane you'll start thinking it would just be easier to own. That's when things get really interesting because you have to decide if this is a hobby or a serious hobby.
*When I got my PP-ASEL, you had two choices - a rigorous, structured 35 hour course or a looser 40 hour minimum course. That's 40 hours of flight time and instruction, not total hours in a classroom. Considering you're a hacker news poster I'm going to go out on a limb and assume you have the disposable income, education and predisposition to dedicate a couple of weeks to doing nothing but learning to fly. If you can manage that I recommend the 35 hour course: it's intensive but the cheapest option because you won't be paying by the hour to train and retrain skills that get rusty by only flying once or twice a week. Flying is very much a mechanical skill and it has to become second nature before you can learn to multitask and take care of all the other things you need to do, otherwise you're perpetually "behind the airplane."
Anyway again if you decide to seriously pursue it I'd say try a discovery flight first and then dedicate a chunk of time to knock it out. It is the fastest way to get it done, which means it's the cheapest way to get it done.