Submission Guidelines

  1. The original paper has to be substantive in nature, with high relevance to and well cited in respective journal. In this regard, we require authors to provide the following information about the original paper (both measures are readily available on the Web of Knowledge):
  2. Having had experience with the Replication Corner for over a year, we find that the most interesting replications extend the original paper in significant ways. As a consequence, a bit more space is sometimes needed to adequately represent what was done and what it means. Therefore, we have decided to increase the word limit to 2000 words.
NOTE: 2000 is a STRICT limit and not a suggested length. Authors are responsible for providing a word count (in addition to the citation count and including in the paper a table comparing their results to those of the paper being replicated). Papers over the limit will be returned to the authors without being reviewed. ​A 100-word abstract and 10 reference can be included.  The replication should include a short introduction on the chosen effect and the difference between the original paper and the replication. This should be followed by a more detailed report of the procedure, summary of the results, and limitations of the replications. In general, replications should be written in a concise and clear writing style.
  1. The replication should include a clear comparison of the results, as well as any difference in the data collection and the methods that were used between the original paper and the replication. The authors are encouraged to provide this comparison in a table.
  2. Figures, tables, the data, and the code used should be submitted and will be posted on the website of the journal, for public use.
  3. We will consider reports of failed replications as well, although we might ask one of the original authors to review the replication attempt. In such cases, a careful examination of quality checks (e.g., attention) becomes an important part of the report.
  4. If the original author reports a significant effect and a second author finds no significant effect, it is always unclear whether the difference in results is a “failure to replicate” or just what one would expect from random draws from a common effect size.  We would like to ask authors to include in their papers or at least an online appendix a meta-analysis including the original Study and attempted replication. An example of such a meta-analysis in the online appendix comes from Chark and Muthukrishnan (IJRM Dec 2013) A good short summary of how to do this is in Rosenthal and de Matteo 2001 Annual Review of Psychology. It is easy to have a situation where one effect size is significant and another is not, but no significant heterogeneity across the studies. If the heterogeneity is not significant, then one can calculate the weighted average effect size and test whether the effect is significant pooling across all the available studies.  If there is no significant heterogeneity but the weighted average effect size remains significant, the original conclusion would stand. If there is no significant heterogeneity and the weighted average effect size is NOT significant, then this calls into question the original finding.  If there is significant heterogeneity, then this raises the question of what is the moderator or boundary condition that explains the difference in results.
  1. In the cover letter, authors should explain why they think their replication is important and provide evidence for it (i.e., citation estimate in the important journals).
  2. The authors have to send along with their submitted replication the paper where the original phenomenon is reported.
  3. One of the important considerations in reviewing the replication is how appropriate is the use of subjects in experimental or empirical replication study.
  4. Before embarking on a replication study on an effect, authors may consult with the editors by submitting a one-paragraph proposal for a pre-study evaluation.