Secular Pinoy

<a href="http://www.nytimes.com/2002/07/07/weekinreview/07KOLA.html" target="_blank">NY Times Article</a> (requires registration, but it's free)

I think the idea of such a journal would be a good idea. We can publish the hundreds of failed paranormal tests, as well as failed tests on alternative medicine. All we need is the cooperation of paranormal/quack testers to have their negative studies tested. (Fat chance!)

I don't know if creationists have ever had independent tests trying to prove their pet hypothesis, and I doubt if they'd be willing to have them published. Hey, if they want the recognition of mainstream scientists, then publish their reasearch in a journal for negative research results, right?

fando

Such a journal has always been an excellent idea. I recall (via anecdote) that Feynman pressed for such a journal for science way back in the 80s. Why it hasn't happened yet is mysterious. And it should be trivial to compile a huge list of null results immediately by harvesting information from all the Sketpic magazines and the like. Not much work is required, just a reference to the published theory or experiment and a summary of the results and conclusions.

Splat

Great idea, old idea, why not?

Three quick answers (IMHO): volume, time, and credit.

Volume: as the article noted, most results are negative, and publication is expensive. E-publishing might help overcome that barrier, but even electronic venues need editors, upkeep, etc.

Time: writing up negative results takes as much time as writing up positive ones. Once something fizzles, it's easiest just to move on. And once they're written, they still need to be peer reviewed, which already consumes immense amounts of the scientific community's time and energy.

I'm skeptical of the NOGO strategy because it seems that it might not distinguish between poor studies that yield negative results simply because they were poorly done, and those that were well done, but wrong. That could shut down avenues of inquiry that shouldn't be shut.

Credit: publications help you get you tenure, promotions, and funding. Until the powers-that-be are willing to recognize negative results, there's no impetus to waste time publishing them. That could change, but it isn't going to happen anytime soon.

Still, this really is something that needs to be done. Another danger of only publishing positive results is that it can select for studies that obtain positive results by accident. E.g., if we do 100 studies to test for some effect and set our statistical significance level at 95%, we should expect 5% of the studies to yield positive results when in fact their is no effect. Guess which 5 will get published?

As for the creationists and the paranormal yoyos, real scientists don't test their hypotheses because the "theory" is simply too absurd to waste time with. Undermining their garbage is important for public opinion and policy, but it doesn't advance scientific knowledge at all. Also, it's not like the people who need to be convinced are reading Science and Nature. That's why people like Gould, Sagan, Dawkins, et al. fight those battles in popular culture venues, rather than scientific fora.

Veil of Fire

Great! Give us the funding and we'll be happy to finally do real scientific testing!

I thought freethinkers were supposed to OBJECT to that sort of telelology. Creationists reject evolution on the same principle, after all. "It sounds/looks stupid, so it CAN'T be true!"

fando

I believe that there are various collections of null results floating around, especially related to paranormal claims. It's just that they're not organized in any meaningful fashion. Organizing them shouldn't be very hard or take too much time. The only problem is funding. A journal of paranormal investigations that turned up nothing is not really useful to science, so the only funding source will be ordinary people and I don't think very many people would concern themselves with such a boring set of data. So a real journal is out of the question. However, there still exists another possibility: Create an online community like DarwinAwards.com or snopes.com and rely on volunteers to populate its database.

Veil of Fire

fando: Who's in charge of quality control, tho?

fando

Veil: If there were an online journal run by volunteers, I'm sure they can use moderation to ensure quality.

Splat

Well, yes, but if you're looking to make a scientific career (from a finite lifetime), you try to pick research topics that are novel enough to get you credit for discovering something new (or problematizing some orthodoxy), but plausible enough to be worth your investment of time.

Psych isn't my field (I'm working from memory of Psych 101, 20 years ago), but as I recall, at one point, paranormal phenomena fit that bill for some psychologists, but it never panned out. Fando's right that it should be pretty easy to collect whatever papers were published and write a review article that the psych community might find interesting enough (if only as history) to publish in a mainstream journal.

The creationists have never served up anything that comes close to being plausible enough to attract serious scientific scrutiny. What would the scientific community gain by testing the current claims of YECs and publishing them in reputable journals? In terms of knowledge production, probably nothing. In terms of changing YEC's minds, probably nothing, since their minds are already made up. The battleground for hearts and minds is, again, in the realm of popular culture.

As for Fando's idea of an online journal with quality control by volunteer moderators: it's a good idea, and that's essentially what the peer reviewed literature (both online and off) is. With few exceptions (Nature, Science...), journal editors aren't paid (you might get a symbolic honorarium); they're volunteers. Ditto for reviewers.

The problem I see is simply not having enough qualified people (PhDs in genetics, for example, who could rigorously review all the failed genetics experiments). Amateur scientists could contribute, but truly rigorous peer review usually depends on a relatively small pool of people specialized enough to be able to pick something apart and do it well (peer review is anonymous, but you can sometimes make very good guesses about who's written what).

The only solution I can see is for the scientific community to make a solid committment to publishing negative results (including giving full credit), increase the number of scientists involved in the process, and decrease the publishing expectations (for + results) on each of them. Probably not the only solution, but it's the best I can come up with!