Why don't researchers publish failed experiments?

"Why don't people publish failures?"

Actually, they do.

  1. Journal of Negative Results (ecology and evolutionary biology)
  2. Journal of Negative Results in Biomedicine
  3. Journal of Pharmaceutical Negative Results
  4. Journal of Interesting Negative Results (natural language processing and machine learning)
  5. Journal of Negative Results in Environmental Science (no issues yet?)
  6. Journal of Errology (no issues yet?)

and so on...

(You might also want to see the Negative Results section of the Journal of Cerebral Blood Flow & Metabolism.)


Null results are hard to publish. They just are. Interestingly enough however, in my field they are not the hardest thing to publish. The general order goes:

Well powered (big) studies that find what people expect
Poorly powered (small) studies that find what people expect
Poorly powered studies that find the opposite of what people expect or null findings
Well powered studies that find the opposite of what people expect

Those middle two categories are where you'll find most "failures", at least in terms of finding a statistically meaningful effect. That being said, there's an increasing push to see these types of studies published, because they're an important part of the literature, and several medical journals have made fairly remarkable steps in that direction - for example, if they accept a paper on the protocol for an upcoming clinical trial, they also commit to publishing the results of the trial (if they pass peer review) regardless of the finding.

When it comes down to it, I think there's three reasons negative results aren't published more beyond "it's hard":

  1. Lack of pay off. It takes time and thought to get a paper into the literature, and effort. And money, by way of time and effort. Most null findings/failures are dead ends - they're not going to be used for new grant proposals, they're not going to be where you make your name. The best you can hope for is they get cited a few times in commentaries or meta-analysis papers. So, in a universe of finite time, why would you chase those results more?
  2. Lack of polish. Just finding the result is a middle-step in publishing results, not the "and thus it appears in a journal" step. Often, its easy to tell when something isn't shaping up to be successful well before its ready for publication - those projects tend to get abandoned. So while there are "failed" results, they're not publication ready results, even if we cared about failures.
  3. Many failures are methodological. This study design can't really get at the question you want to ask. Your data isn't good enough. This whole line of reasoning is flawed. Its really hard to spin that into a paper.

Successful papers can be published on their own success - that is interesting. Failed papers have the dual burden of being both hard to publish and having had to fail interestingly.


It is not completely true that failures are not published. Lack of signals, or lack of correlation are published. The point is that everything that pushes knowledge forward is worthy of publication. That said, there are other factors you have to keep into account

  1. some failures are methodological, that is, you are doing something wrong. That is not a scientific signal. it's something you have to solve.
  2. knowing what doesn't work gives you a competitive advantage against other research groups.
  3. negative signals almost never open new fields. If they do, it's because they steered attention to find a positive signal somewhere else. You don't open a new cancer drug development if a substance is found not to have an effect. You close one. For this reason, negative papers generally don't receive a lot of attention, and attention from peers is a lot in academia.