Does Tweeting Improve Citations? One-Year Results from a Randomized Trial

He published Freakonomics together with Stephen Dubner then went back into academia while Dubner went down the podcaster route.

15 years later, Dubner has reached millions of people with his work and Levitt recently started a podcast and soft quit academia after realizing that one of his best papers after years of research and hard work got 3 citations. He assumes fewer than 20 people will ever read that work.

One of the causes of the slowdown (even if marginal) of science and research in general is the administrative and social media overlays that were added to them.

Two distributions can overlap a lot and still have a low p-value for anova etc if there are enough data points.

Just ignoring the stats though the results seem pretty solid. 3x the number of citations with ~100 observations. If the tweets were truly randomly assigned and there aren't big outliers driving results (big ifs, not gonna go deep enough to find out), their conclusion should be fine.

It’s also perfectly justified to search for papers solely based on how influential they’ve been in your field. Papers which trend on Twitter (perhaps long-awaited results) are already having an impact on labs, probably being discussed in journal club and shaping the trajectory of the field.

Related work sections aren't typically thorough, that's what lit surveys/reviews are for. It's also not a new phenomenon that highly cited work gets cited because it's highly cited, i.e. work is cited simply because it's more readily noticed. It's furthermore common for the same research to arise independently in different fields, with different terminology, i.e. bodies of research are somehow not found. So, no need to be scared, it's nothing new.

A thorough literature search is mostly needed to avoid claiming as novel innovations/discoveries things that others have previously published (even if you more-or-less ended up discovering it independently). Secondarily, it also helps you avoid ratholes that others have stumbled into (although the reduced incentives for publishing negative results makes this less effective).

However, citations of papers encountered on — and recalled from — social media are more likely to be for the inspiration they provided for the line of inquiry or methodology in your paper (even if that inspiration was negative, ie. you're refuting someone's work or conclusions).

This effect would have to be replicated in some form to understand its boundaries better but if it were to generalize I think it has much broader implications than just Twitter and thoracic surgery.

There have been lots of bibliometric studies of citation impact etc. but this is one of the first times I've seen an actual experimental study in academics of this sort of thing, and it confirms what a lot of people have been experiencing, which is that actual scientific quality is only part of the reason why some work gets lots of attention. Replace twitter with other forms of social networking and the implications become clear.

There's ups and downs to this: maybe some underappreciated work would get more attention if it's marketed more, for example. But it speaks to the role of things outside of the domain of study per se (not sure what the term for this would be -- something like nondiagetic but for scientific content).