This is a guest post written by Sigma Xi President Tee Guidotti.
Vox, the popular online culture and news content provider (“magazine” is so passé!), has just published “The 7 Biggest Problems Facing Science, According to 270 Scientists,” by three of its top science writers. Our aspiring students, graduate students, postdocs, junior faculty, future administrators, and many others in the Vox demographic are reading that “science is in big trouble” and “many scientists are afflicted with a serious case of doubt—doubt in the very institution of science.”
Briefly, Vox Media did a survey asking 270 scientists (including some unstated proportion of graduate students) from “all over the world” one question: “If you could change one thing about how science works today, what would it be and why? The survey obviously has big methodological and sampling problems but the findings, reordered here for logical flow, will not surprise career scientists.
1. “Academia has a huge money problem.”
Well, duh. Vox correctly identifies universities as the pinch point, because they are the home of investigator-initiated scientific research and the boot camp for basic training in research. Vox describes the current funding situation well but stops short of the deeper implications for academia: Universities are essentially rental units providing housing and infrastructure to cottage-industry research enterprises that depend for survival on sponsors, chiefly the U.S. federal government. When support falters or is unpredictable, it shakes the entire system: research faculty appointments, evaluation and promotion, continuity of research and training, and return on investment in research infrastructure. Investigators naturally become averse to taking risks. Arguably, research grants from the major funding agencies are no longer a solid foundation for a serious academic career. In biomedical sciences, especially, the system now leaves most talented scientists behind and often strands them in mid-career, diverting them from other careers or fields where their contributions might be greater.
2. “Too many studies are poorly designed. Blame bad incentives.”
True again. A significant portion of published science is poor quality or misleading to uncritical readers, usually junior investigators or lay readers. The volume of mediocre work clogs the literature.
The pressure to publish is intensifying worldwide because research activity is equated with quality and value in faculty advancement. At present, the only practical way to evaluate and grade research on a mass scale is for faculty to submit their work in the form of manuscripts for peer review and to use publication as a proxy measure for quality.
3. “Peer review is broken.”
Not exactly news. The fundamental problem behind mediocre studies debasing the literature is that there is only one acceptable way to evaluate quality, and that is peer review. STEM publications have become so specialized that peer review can only be done by a colleague in the same or a closely related field, which leads not only to conflicts of interest but to reviewer fatigue, cursory reviews, and bias born of heuristics (mental shortcuts), because few reviewers have time to look at the problem deeply from the investigator’s nuanced point of view. Other problems of peer review are equally manifest and well known. (As a journal editor, managing peer review is my single biggest challenge.)
4. “Too much science is locked behind paywalls.”
Yes, there are commercial barriers to accessing much important science: user and pay-per-view fees, subscription costs, licensing fees and royalties, proprietary data restrictions, patents, and other obstacles. The open access movement is an attempt to break through all this, a revolution with the rallying cry “information wants to be free!” Sigma Xi supports open access. Still, before universal open access can flourish unfettered, important issues of the gatekeeper roles of editors, the quality assurance function of reviewers, and the interests of authors remain to be sorted out.
5. “Replicating results is crucial. But scientists rarely do it.”
Kind of true, but strict replication is not always practical. It was once common for scientists to replicate their experiments themselves several times before publishing.
There are only two situations today in which scientists would want to replicate someone else’s study exactly and that is 1) when they are starting a new line of investigation and have to master the technique, and 2) when the original study is already suspect or incompatible with new evidence. Otherwise, replication is time and funding lost. More often today, scientists rely on convergent evidence, in which a study using different methods produces a similar result that supports the finding.
The main factor that discourages replication today, however, is probably the strong management pressure and policies to avoid “duplication,” which seems wasteful and unnecessary to managers, funding agencies, and Congress.
6. “Science is poorly communicated to the public.”
Absolutely, undeniably true. Sigma Xi has been addressing this problem for years through its magazine, American Scientist, and through events run by its chapters that bring science to the public.
7. “Life as a young academic is incredibly stressful.”
Well, yeah. No argument there.