Open Source is particularly useful for the long tail. That particular library that is very relevant to you and maybe 20-50 other people in the world. It makes sense to join forces and make it better - but it will never be absolutely mainstream.
Secondly, I say your point 'partially true for mainstream things' because even in mainstream things, there will always be customizations/particularities that again are useful to you and 30-50 other people in the world - thus you join forces (e.g., a obscure linux module).
Agree with you that 99.9% of the people will not
It's all a matter of long tails. Without open source, all long tail efforts would be each independent person doing it for themselves (when you have 30-50 people in the world that should be sharing!)
Although the modern web is making it viable for small teams w/little capital to go after long-tail in a commercial way (many YC startups are a testament to this) --- although I don't imagine them doing this for a 30-50 people target market.