Nature opens its anti-publications-bias format to all fields, where up to 60% of hypotheses fail review

Nature issued new guidelines expanding the types of research and research fields that can use the Registered Report format. 

Registered Reports are a format for empirical articles in which the hypotheses, methods and planned analyses undergo peer review before the research is conducted. Previously, Nature only considered these reports for confirmatory research in cognitive neuroscience and behavioral and social sciences. 

Now, Registered Reports will be welcome across all fields Nature publishes. In the reports, authors propose a topic of study, explain its importance and describe plans for data collection and analysis. This is peer-reviewed before data is collected or accessed. If the report is accepted after the review, the journal commits to publishing the study, regardless of the outcome. 

This could increase the number of null results published, helping to counteract publication bias. Null results can provide an important counterweight to positive findings, ensuring conclusions are not accepted too readily. 

“The idea is that if your research question is important, the answer is going to be important too, no matter if it is a yes or a no,” said Mary Elizabeth Sutherland, a senior editor at Nature. 

A 2019 study published in PLOS Biology found that the experimental data did not support 60.5% of the preregistered hypotheses in Registered Reports. This is substantially higher than the estimated 5% to 20% reported in the traditional literature. 

“Not all null results and replications are equally important or informative, but, as a whole, they are undervalued. If researchers assume that replications or null results will be dismissed, then it is our role as journals to show that this is not the case,” Nature stated in an article. 

In this approach, reviewers can help shape a study by making suggestions instead of providing comments at the end. It also ensures that all results are published, including negative or inconclusive results. This may help to reduce P hacking, where researchers conduct many statistical analyses until they get a significant result. 

“Because peer reviewers have the power to change what’s actually happening, it feels like it makes people more constructive. I feel that difference has the potential to shift the tenor of peer review and make people feel like peer review is designed to be more helpful,” said Sutherland. 

There may also be personal benefits to researchers using Registered Reports. They reduce uncertainty around whether the study will be published, making it easier to plan future work and grant applications, Rose O’Dea, a postdoctoral researcher in Berlin, told Nature Communications in an interview. 

One challenge of Registered Reports is that researchers are locked into their method of analysis. To help mitigate this, the Nature guidelines allow researchers to carry out unforeseen exploratory analyses, as long as they are clearly identified as such, are justified and reported separately from the main results. The results from these analyses cannot be the basis of the paper’s conclusions. 

“Expanding Registered Reports to more disciplines and research types reflects our conviction that these standards can elevate all forms of inquiry. This should make science more rigorous and transparent — and put a greater emphasis on work that meaningfully advances knowledge,” Nature said in an editorial. 

Registered Reports can be more time-consuming than traditional methods, which could make them unsuitable for researchers on a tight timeline. Some critics also say it repeats the grant proposal process. There could be a way to streamline this process in the future, Sutherland said. 

How to submit a Registered Report

All Registered Reports must initially be submitted as a presubmission enquiry that contains the research questions, an explanation of why the project is suitable for the report format and a general description of the intended data and proposed analyses. 

Once a presubmission enquiry is invited for submission, it must be written up according to the Stage 1 Registered Report template. The template contains a description of the research questions and background literature, hypotheses, experimental procedures, analysis pipeline, a sampling plan and pilot data. 

The Stage 1 report should also include a cover letter with a statement confirming all necessary support and approvals are in place, an anticipated timeline and statements regarding the data and publishing permissions.

In the second stage, the authors prepare and resubmit their manuscript for full review with a cover letter that includes statements regarding the availability of data and a summary of the additional exploratory analyses that were conducted. The Stage 2 manuscript must also contain a link to the registered protocol.