Publication Debate

Here's a story that gives an interesting illustration of two conflicting perspectives on the role of scientific publication.

Wildfire logging debate heats up Nine scientists wrote a letter to Science asking the journal to withhold a one-page article on the potential risks of post-wildfire logging, arguing the article was short on qualifiers and context. But some forestry scientists say they support the conclusions, and last week, the journal published the paper. ... Editors at Science "encouraged us to submit a technical comment," [letter co-signer John] Sessions said, which he and his colleagues are preparing now.

The letter writers appear to hold to an older, more Hierarchist, view of scientific publication. In this perspective, scientific publications -- particularly high-status and high-public-profile ones like Science -- should be records of scientific conclusions. Controversies should be hashed out within the scientific community, and only results of agreed-upon high quality should be published. A journal article is like a small brick of reliable knowledge added to the edifice of scientific understanding. By this logic, it is reasonable to take extreme measures, such as petitioning a journal to block a paper you disagree with, lest a weak brick be incorporated into the structure.

The alternative view, supported by Science's decision to publish the controversial paper and request a rebuttal article by the letter writers, is that the scientific literature is a debate forum. Rather than exhaustively vetting the quality of results before they appear, this alternative perspective sees publication as the site for hashing out these issues. Here the filters on what gets published should be weaker, meant only to weed out obvious and egregious errors -- a standard that the letter writers' complaint doesn't meet, given that their central claim seems to be that the paper lacks documentation of contextual factors that would delimit the scope of generalizability of the paper's results. Such issues are more appropriate to be worked out in a public debate and through further research.

Am important complication here is that in the wider public debate, scientific publication is too often treated as if it operates under the first model. Already, critics of salvage logging have cited this one paper as proof that salvage logging should be ended -- after all, it was published in Science. This is not to say that advocates shouldn't support their arguments with debatable publications, but rather that they must be prepared to defend their scientific merits instead of just stopping with "if Science says it, it's so." A more generous view of the letter writers' perspective might be that if the public is going to treat journal articles like bricks of truth, then scientists have a responsibility to approximate that ideal as best they can, lest their work be misappropriated and conflicting papers weaken faith in the scientific enterprise. This, however, is only a short term solution. Where policy relevance is high and the ability to do controlled experiments is low, simple cumulative knowledge-building is especially elusive. Thus the danger of substituting orthodoxy for reliable consensus is high. What is necessary is the harder work of shifting public attitudes, from a quasi-religious reliance on scientific pronouncement to a fuller understanding of how science actually works.