Structurally it seems odd that journals are still organized as periodic papers stitched together. It would be better in a digital system for a study webpage to have an addendum underneath that includes links to replications under the same journal for better organization.
Who really is out there reading every single item published by a journal? And who will stop doing that if they occasionally publish a replication?
The system of replications reducing the impact factor should be done away with unless the original paper is adjudged as outright fraud. Since the replication is also research, replications of non-fraud papers should also be counted for estimating impact factors.
Just one anecdote about JPSP and Eliot Smith -- way back when (ca 2009), we were replicating work by Diederik Stapel. Of course, we were finding a whole bunch of nothing.
We know what transpired in 2011.
We then decided to collect all the replications we had done of the specific paper (we still had the data and materials of 12). We went back to JPSP, where the original was published.
Eliot Smith's response was, indeed, that JPSP did not accept replications (at the time). When we pointed out that it was technically not a replication, as the original never existed, his response was that the original was interesting, because the results were significant. So, we instead had to go to Acta Psychologica (https://psycnet.apa.org/record/2015-16144-004).
(now, "fun fact" the other author's former supervisor would often say that "she had shit on her hands" as she was unable to replicate Stapel. Such were the times if you could not get your effects to be significant)
Structurally it seems odd that journals are still organized as periodic papers stitched together. It would be better in a digital system for a study webpage to have an addendum underneath that includes links to replications under the same journal for better organization.
Who really is out there reading every single item published by a journal? And who will stop doing that if they occasionally publish a replication?
The system of replications reducing the impact factor should be done away with unless the original paper is adjudged as outright fraud. Since the replication is also research, replications of non-fraud papers should also be counted for estimating impact factors.
Another great post.
Just one anecdote about JPSP and Eliot Smith -- way back when (ca 2009), we were replicating work by Diederik Stapel. Of course, we were finding a whole bunch of nothing.
We know what transpired in 2011.
We then decided to collect all the replications we had done of the specific paper (we still had the data and materials of 12). We went back to JPSP, where the original was published.
Eliot Smith's response was, indeed, that JPSP did not accept replications (at the time). When we pointed out that it was technically not a replication, as the original never existed, his response was that the original was interesting, because the results were significant. So, we instead had to go to Acta Psychologica (https://psycnet.apa.org/record/2015-16144-004).
(now, "fun fact" the other author's former supervisor would often say that "she had shit on her hands" as she was unable to replicate Stapel. Such were the times if you could not get your effects to be significant)