There's lots of approaches (as you note), but if I had to boil them down to a rule of thumb it would be this:
give away functionality, charge for convenience.You love our app, but want an MSI installer so you can deploy it on your network more easily? You can roll your own by checking out our source code here... or you can pay us $X for this already-made-and-thoroughly-tested installer.
You love our app, but want to learn more about how it works? You can browse this repository of 15,000 Doxygen-generated API pages... or you can pay us $Y for this beautiful book that explains everything in plain English.
You love our app, but want someone who can support it if something goes wrong? You can spend ten years turning someone on your staff into an expert on it... or you can pay us $Z for this support contract, and get access to our experts.
This approach can feel counter-intuitive to geeks, because we value functionality more than convenience, so it feels "wrong" to charge for the things we see as needless frills. But in general, people put a lot of value on convenience, and they are more than willing to pay for it if you're willing to provide it.
Sometimes people try the opposite approach -- charging for functionality. (i.e. "Try the open source edition of our app, or pay $X for the Premium Edition and get custom fields!") This rarely works.