1) Dual licensing 2) Charge for commercial support 3) Sell hosted versions (might not apply to you) 4) Advertising?
"Because making something people want is so much harder than making money from it, you should leave business models for later, just as you'd leave some trivial but messy feature for version 2. In version 1, solve the core problem. And the core problem in a startup is how to create wealth (= how much people want something x the number who want it), not how to convert that wealth into money."
Cheers
For an open-source library, the right-to-fork and to use without paying licensing fees means many of the options used by other startups which defer revenue questions in favor of building an enthusiastic userbase are already off the table for you. You can't easily withhold your product unless fees are paid; you can't add advertising; you may not be in a position to sell excludable complementary offerings.
You could try a dual-licensing structure like Sleepycat BerkeleyDB -- free for other open-source projects, pay if you want to use in a closed-source application. However, that presupposes a large closed-source commercial market for the software your library-users make, requires some capability for closing big corporate sales, and it may take years of proven library quality before becoming the solid/safe choice for companies to embed in their own products.
And the unstated assumption of that interpretation is that it is impossible to make money without coercion.
Excludability ≠ coercion.
An advertising-supported site can exclude non-payers from using their advertising inventory. Money is made without coercion (and free/open-source software might be provided as an audience generator).
A consulting/services/support business can refuse to provide their service to non-clients. Money is made without coercion (and free/open-source software might be central to the delivery of services or expression of expertise).
A paid-subscription website can exclude non-subscribers. Money is made without coercion (and free/open-source software might be provided as a complement to the paise service).
The OP's proposal of angel-funding an open-source library presents a challenging case, unless some excludable complementary product can be offered. Only the dual-licensing strategy could vaguely be said to include coercion, if you believe enforcement of copyleft or copyright is coercive. And as I noted, that approach requires overcoming other market/sales/staying-power difficulties.
If you have other ideas for turning a open-source library into a money-making non-coercive business, please share -- especially if they somehow work without excludability. I'm very interested in this topic.
Your points about service businesses are well-taken, though I think this is a hard and uncertain road to turning a profit.
As for turning free software into a money-maker, I think not-for-profit is the stable state of a perfectly efficient software industry. That doesn't mean programmers couldn't be well paid, but it's likely nothing that would be terribly attractive to VC.