Obviously science and the process of science have been on my mind.
My fellow neuroscientists Jason Snyder and Björn Brembs and I got together for lunch to talk about... lots of stuff, actually... but I came away with a few questions that seemed to have empirical answers.
Before I jump in though, if you haven't seen Björn's talk What's wrong with scholarly publishing today? check out the slides at the end of this post. They're packed with some mind boggling data about the business of peer-review.
At some point during our lunch, Retraction Watch (which is an amazing site), came up, and ultimately inspired two questions:
- Which journals have the most retractions?
- Which biomedical fields have the most retractions?
There was one issue: every article--regardless of scientific field--for the general science journals (Science, Nature, PNAS) are indexed in PubMed. So if an article (or a dozen) about semiconductors (see: Jan Hendrik Schön) was retracted from Science, it would still show up in this analysis. The result was inflated biomedical retraction counts for those journals, so I had to manually adjust counts down by removing non-biomedical retractions (just to put everything on par, since PubMed doesn't index non-biomedical peer-review journals).
Here are the results for the 1922 retractions across 796 journals:
PNAS (59 retractions) and Science (52) lead the pack, followed by J Biol Chem (40), J Immunol (33), and Nature (31).
Next I counted words that appeared in the titles of the retracted articles to get a feel for what kinds of papers are being retracted. Here's all words that appear at least 50 times in paper titles:
- cells (189)
- activity (154)
- effects (152)
- human (148)
- patients (136)
- protein (108)
- factor (104)
- gene (103)
- expression (102)
- receptor (96)
- study (81)
- cancer (70)
- treatment (57)
- surgery (54)
- disease (54)
- DNA (53)
- virus (50)
At first blush, it looks like cell/molecular/micro biology represents a big chunk of the retractions (cells, protein, factor, gene, expression, receptor, DNA, virus), but human patient research isn't much better off... (human, patients, surgery).
I've heard the argument before (sorry, can't remember where) that fields where the data is more difficult to collect and replicate are more prone to shady research practices... I'm not sure if that's exactly being reflected here, but the exercise was an interesting one.