MetaROR (MetaResearch Open Review) is a platform for open peer review and curation of published metaresearch articles. Unlike traditional journals, MetaROR uses the publish-review-curate model. Before submitting to MetaROR, authors publish articles as preprints, working papers, or reports. After submission, MetaROR provides open review and curation. Curation happens through posting a public editorial assessment by the article’s Managing Editor (ME) in collaboration with the Editor-in-Chief (EIC) who assigned the article to the ME.

MetaROR welcomes the submission of articles in all fields of metaresearch (or research on research), including studies related to higher education, history of science, philosophy of science, science and technology, science of science, scientometrics, and sociology of science. Topics of interest include, but are not limited to, the study of scientific practices, research funding, communication of research, research evaluation, reproducibility/replicability of research, the scientific workforce, and research methods. We consider both basic metaresearch and metaresearch with a more direct focus on science policy and practice. We also welcome articles that translate results from basic research into policy-relevant insights. We review and curate both qualitative and quantitative research and literature reviews.

MetaROR does not accept or reject articles following peer review, unlike traditional journals. The review and curation process produces two outputs: i) public review reports that comment on the article’s strengths and weaknesses, indicate whether the reported results justify the claims and conclusions, and offer ideas for improvement, and ii) a public editorial assessment that summarises the article’s main strengths and weaknesses and any significant reviewer disagreements, and provides an overall assessment of the article’s quality and its contributions to the field.

After the review process is complete, all outputs are published on the MetaROR platform. A DOI is assigned to each review report, and reviewers retain the copyright. Reviewers consent to publication of review reports under a Creative Commons Attribution License. Reviewers are strongly encouraged to sign their reports, but reports can be published anonymously. The ME is required to sign the editorial assessment.

Prior to the publication of review reports and editorial assessments, reviewers will be offered an opportunity to read other reviewers’ reports and edit their own reports. Reviewers will also have an opportunity to review and comment on the editorial assessment before publication. Once the reviews and editorial assessment are finalised, authors are given an opportunity to review the materials and correct any factual errors.

Review reports are due within six weeks of the request. MEs will aim to conclude the collaborative discussion between reviewers within three weeks of receiving all review reports. MEs will also aim to publish all review reports along with the editorial assessment on the MetaROR platform within two weeks after sending materials to the authors for review and correction. Authors are invited to submit responses to the review reports and editorial assessment. After review, responses are posted on the MetaROR platform next to the review reports and editorial assessment.

After the review process is complete, authors may submit a revised article to MetaROR. The revised article must be publicly available on an appropriate preprint server or some other repository and be accompanied by a response to the reviewers’ comments. The ME will decide whether the revised article warrants a new review process. Reviewers will be asked to evaluate the revision and submit a revised review report. The ME will update the editorial assessment in collaboration with the reviewers and the assigning EIC.

MetaROR encourages professional, constructive, rigorous, and respectful review reports. Review reports should be precise and clear. Reviewers should be open to new approaches. They should avoid unconstructive, ambiguous, and unsupported comments. Reviewers who lack the expertise to review some aspect of the article should explicitly state this in their review report. Reviewers should agree to review only if they will be able to submit a review report within six weeks of the request.

We recommend including:

• a clear description of the article’s distinctive contribution to the existing literature
• a list of major strengths and weaknesses of the conceptual framework, methods, and interpretations and discussions of results
• Are the ideas clearly communicated?
• Is the relevant existing literature described and discussed? Are the references appropriate?
• Are all research methods appropriate, rigorous, and clearly described?
• Are the results clearly presented? Are graphs easily readable and clearly labelled? Do they contain descriptive captions?
• Are the results discussed in the context of previous findings? Are differences from previously reported findings clearly described?
• Are interpretations and conclusions consistent with the empirical materials and data?
• Are the limitations of the study and their implications for interpreting the results clearly described where applicable?
• suggestions for improvements
• any issues related to data and code availability, research ethics, or other issues related to lack of adherence to MetaROR policies (see Author Guidelines)

Review reports assess articles in their current published form. Reviews should not comment on the appropriateness of the article for publication elsewhere or speculate about which journal might publish it.

Reviewer anonymity. Reviewers are strongly encouraged to sign their review reports, but they may choose not to sign and to remain anonymous. If a reviewer chooses to sign a report, the reviewer’s identity will be disclosed to the article’s authors and other readers of the review report. If a reviewer chooses to remain anonymous, the reviewer’s identity will be disclosed only to MetaROR’s editors and, if the reviewer grants permission to the ME, to other reviewers during MetaROR’s collaborative discussion phase. In addition, if the authors subsequently submit the article to a journal, and the editors of the journal wish to make use of the review reports published by MetaROR, the MetaROR ME will disclose to journal editors the identity of all reviewers, including those who chose not to sign their MetaROR review report. MetaROR MEs are required to seek permission from anonymous reviewers to disclose their identities only when the journal intends to disclose the identity of reviewers to the authors and/or readers. If a reviewer chooses not to sign a review report, MetaROR editors and other reviewers must treat information about the reviewer’s identity as strictly confidential in cases where anonymous reviewers agree to share their identities with other reviewers during the collaborative discussion phase.

Competing interests. MetaROR requires reviewers to disclose in their review reports any competing interests that could lead them to be positively or negatively disposed toward an article. Competing interests include any role, relationship, or commitment that presents an actual or perceived threat to the integrity or independence of the review, including but not limited to:

• being a current collaborator with one or more of the authors,
• having published a scholarly work with one or more of the authors during the previous five years unless the co-authoring group is unusually large,
• having worked in the same department as one or more of the authors during the previous five years,
• having a close relationship with one or more of the authors, including previous advisor, mentor, or student,
• expecting a direct financial interest or other professional benefit from the review, and
• being in any other relationship with one or more of the authors that might compromise the reviewer’s objectivity.

Those invited to review must recuse themselves if they feel they cannot offer an impartial review. If no competing interests exist, the review report must include an explicit statement as such.

Misconduct. Reviewers should not allege misconduct in their review reports. If a reviewer is concerned about misconduct, the reviewer should inform the ME. In consultation with the assigning EIC, the ME will investigate the reviewer’s concerns and take appropriate action. MEs will follow Committee on Publication Ethics (COPE) guidelines where possible.

Co-reviewing. MetaROR supports co-reviewing, where an invited reviewer reviews an article in collaboration with a junior colleague. Co-reviewing can provide a valuable learning experience for junior researchers and contribute to a sustainable, high-quality peer review system. In the case of co-reviewing, the reviewer must disclose the identity of the co-reviewer to the ME. The invited reviewer and the co-reviewer should agree on the review’s content. MetaROR’s competing interests policy applies to both reviewers. The invited reviewer should be the main point of contact in discussions between the reviewers and the ME. However, the invited reviewer can confer with the co-reviewer during these discussions.