Many scientific articles are published daily, sometimes first as pre-publications on sites such as MedRxiv and BioRxiv and then in scientific journals. Some of these describe poor research or are even based on fraud.
Pre-publications can be helpful to quickly share results, as the regular publication process of a manuscript can take a long time. The disadvantage of pre-publications is that they are not peer-reviewed, while peer review should filter out errors in an article or overall poor research. The normal peer-review process is also not watertight, so every now and then, articles that are not worthy of publication slip through the cracks.
How can you tell whether an article contains solid research, mostly poor research (or even fraud), or something in between? And where can you find out whether other researchers found irregularities in a publication? Before you invest a lot of energy in experiments that build on other people's results, or if you have doubts about an article, finding out what others have said about the research can be helpful.
Preprint servers
A manuscript published on a preprint server such as MedRxiv, BioRxiv, or Preprints often contains links to blogs written about it or mentions in the media (example). These sites also include links to comments on other sites (under 'community reviews'; example) such as PubPeer. In addition, other researchers can leave comments (example). In this way, questionable results or poor research are often quickly identified.
Articles published in a journal that were previously posted on a preprint server usually contain a link to the preprint—this way, you can find previous comments.
Public peer review
- PubPeer
On the website PubPeer, people can comment on articles published as a preprint or in a journal. The corresponding author receives an email when a comment is posted. Often, these comments are minor points that can be corrected or explained by the authors, and sometimes the comments lead to the correction of the article.
However, there are also cases where serious problems are found. Examples are the various publications on hydroxychloroquine for treating COVID-19, such as that of Gautret et al. That article had over 70 comments on PubPeer pointing out many issues with inclusion, endpoints, analyses, and results. The journal has since retracted that article.
- PREreview
PREreview is also a website where reviews are posted, but only of pre-publications. The reviews on this website are more like journal club reviews in that they often discuss the entire article, not just the flaws. There seems to be a lot of effort put into the reviews.
In addition, you can request a review of a specific manuscript here, which is helpful if you cannot assess the manuscript yourself.
Poor research, sloppiness, or fraud?
Scientist Elisabeth Bik works as a science integrity consultant, and her mission is to identify duplicate - or otherwise manipulated - images in scientific articles. She reports these questionable images on the website Science Integrity Digest.
In some journals, image duplication is more common than in others. In an analysis of articles published in 40 journals, Bik found that 0.3% to 12.4% of articles contained duplicate images. High percentages are probably due to a combination of sloppy or malicious researchers and a sloppy or missing peer review process.
Bik also brings to light other scientific integrity problems, ranging from misconduct in animal experiments to publications based on poor research in general.
Background stories about poor research
For Better Science is a website focusing on the integrity of biological and medical research. It contains many well-researched stories about dubious scientists.
Retraction Watch
The Center for Scientific Integrity (CSI) maintains the Retraction Watch website. This website contains all published articles that have been, or should have been, retracted. It also includes background stories and interesting lists, such as the top 10 of frequently cited but retracted articles, and a list of retracted articles by Nobel Prize winners. If you find this an interesting topic, you can subscribe to their newsletter.
Poor research – Other sources
If you know of other good websites that can help identify poor research, please let me know so I can add them.
Recent Comments