Scientists have to eat and pay the bills, too. Unless you're really rich, it's tough to do much serious science in your spare time on weekends, paying for equipment from your own pocket. So the idea is that a scientist gets a job that pays him/her to do science. The system rewards scientists who advance knowledge and penalizes those who do not.
How do we determine whom to reward? It is extremely difficult to evaluate most scientists' work unless you are an expert in the same field. So we let peer reviewers evaluate the work, and reward scientists for publications in prestigious peer-reviewed journals.
The problem is that these incentives work against some kinds of sharing.
It's not all bad. A scientist whose work never sees the light of day is fired; no publications means no job. But sharing of ideas apart from those that are necessarily published is something the system tends to punish.
What to do about it? It's easy to propose some radical revision of the system. But we still have to deal with the question of how to evaluate the work of someone whose ideas can only really be understood after a decade of specialized study.
Edit: I'd love to actually, but perhaps I'll wait for the post scarcity, star trek economy to become mainstream.
But the reality is that both the career advancement and funding for further experiments is almost entirely based on the number of publications in prestigious journals.
And effort put into collecting data is not by itself publishable: you must add analysis, and very often analysis that successfully confirms a new hypothesis or invalidates previous beliefs to publish. Journals prefer to publish exciting results, and "we ran experiment X, and found no significant effects" is usually rejected.
Fixing this is desperately needed, but so far nobody has come up with a solution that would actually fix this.
A middle ground would be to require data publication after a certain time period. For example, NASA and ESA release original data 6-12 months after collection. Releasing original data together with the paper also works.
Wouldn't rewarding people for `cites' of their dataset be a way to fix this?
Ie if someone uses your data, they have to cite you, and that's just as good as someone citing an analysis paper.