- See Also
-
Links
- “The Early Days of Peer Review: 5 Insights from Historic Reports”
- “Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews”, Liang et al 2024
- “Can Large Language Models Provide Useful Feedback on Research Papers? A Large-Scale Empirical Analysis”, Liang et al 2023
- “ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews”, D’Arcy et al 2023
- “Saving Time and Money in Biomedical Publishing: the Case for Free-Format Submissions With Minimal Requirements”, Clotworthy et al 2023
- “Comparing Analysis Blinding With Preregistration in the Many-Analysts Religion Project”, Sarafoglou et al 2023
- “Nobel and Novice: Author Prominence Affects Peer Review”, Huber et al 2022
- “Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment”, Cortes & Lawrence 2021
- “An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports”, Scheel et al 2021
- “Honest Signaling in Academic Publishing”, Tiokhin et al 2021
- “Peer Review As an Evolving Response to Organizational Constraint: Evidence from Sociology Journals, 1952–2018”, Merriman 2020
- “Expert Consensus Procedure (ECO): Facilitating Robust Scientific Outputs”, Kekecs et al 2020
- “Are Peer-Reviews of Grant Proposals Reliable? An Analysis of Economic and Social Research Council (ESRC) Funding Applications”, Jerrim & Vries 2020
- “Is Scholarly Refereeing Productive (at the Margin)?”, Hadavand et al 2020
- “The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases”, Schäfer & Schwarz 2019
- “Registered Reports: an Early Example and Analysis”, Wiseman et al 2019
- “The Sociology of Scientific Validity: How Professional Networks Shape Judgement in Peer Review”, Teplitskiy et al 2018
- “Scientific Autonomy, Public Accountability, and the Rise of ‘Peer Review’ in the Cold War United States”, Baldwin 2018
- “Low Agreement among Reviewers Evaluating the Same NIH Grant Applications”
- “Uncertainty and Individual Discretion in Allocating Research Funds”, Goldstein & Kearney 2017
- “Can Results-Free Review Reduce Publication Bias? The Results and Implications of a Pilot Study”
- “To Apply or Not to Apply: A Survey Analysis of Grant Writing Costs and Benefits”, Hippel & Hippel 2015
- “Protocol Review at The Lancet: 1997–2015”, Lancet 2015
- “A Two-Step Manuscript Submission Process Can Reduce Publication Bias”, Smulders 2013
- “On the Time Spent Preparing Grant Proposals: an Observational Study of Australian Researchers”, Herbert et al 2013
- “A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants”, Bornmann et al 2010
- “Peer-Review in a World With Rational Scientists: Toward Selection of the Average”, Thurner & Hanel 2010
- “Telescope Time Without Tears: A Distributed Approach to Peer Review”, Merrifield & Saari 2009
- “Science in the 21st Century: Social, Political, and Economic Issues”, González-Álvarez 2008
- “Is Tenure Justified? An Experimental Study of Faculty Beliefs about Tenure, Promotion, and Academic Freedom”, Ceci et al 2007
- “Effect of Blinded Peer Review on Abstract Acceptance”, Ross 2006
- “Effects of Editorial Peer Review: A Systematic Review”, Jefferson 2002
- “A Letter from the Frustrated Author of a Journal Paper”, Glass 2000
- “Effect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports: A Randomized Controlled Trial”, Godlee 1998
- “Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation”
- “Reflections After Refereeing Papers for NIPS”, Breiman 1995
- “How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists”, Gans & Shepherd 1994
- “The Influence of Prior Beliefs on Scientific Judgments of Evidence Quality”, Koehler 1993
- “Reviewer Bias”, Ernst et al 1992
- “The Philosophical Basis of Peer Review and the Suppression of Innovation”, Horrobin 1990
- “The Story of the NIH Grants Programs”, Strickland 1989
- “Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System”
- “Editorial [EJP Editorial on Registered Reports]”, Johnson 1975b
- “Models of Control and Control of Bias”, Johnson 1975
- “A Proposal for a New Editorial Policy in the Social Sciences”
- “Statistical Modeling: The Two Cultures”, Breiman 2024
- “Real Peer Review Has Never Been Tried”
- Sort By Magic
- Miscellaneous
- Bibliography
See Also
Links
“The Early Days of Peer Review: 5 Insights from Historic Reports”
The early days of peer review: 5 insights from historic reports
“Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews”, Liang et al 2024
“Can Large Language Models Provide Useful Feedback on Research Papers? A Large-Scale Empirical Analysis”, Liang et al 2023
“ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews”, D’Arcy et al 2023
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews
“Saving Time and Money in Biomedical Publishing: the Case for Free-Format Submissions With Minimal Requirements”, Clotworthy et al 2023
“Comparing Analysis Blinding With Preregistration in the Many-Analysts Religion Project”, Sarafoglou et al 2023
Comparing Analysis Blinding With Preregistration in the Many-Analysts Religion Project
“Nobel and Novice: Author Prominence Affects Peer Review”, Huber et al 2022
“Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment”, Cortes & Lawrence 2021
Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment
“An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports”, Scheel et al 2021
An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports
“Honest Signaling in Academic Publishing”, Tiokhin et al 2021
“Peer Review As an Evolving Response to Organizational Constraint: Evidence from Sociology Journals, 1952–2018”, Merriman 2020
“Expert Consensus Procedure (ECO): Facilitating Robust Scientific Outputs”, Kekecs et al 2020
Expert Consensus Procedure (ECO): Facilitating Robust Scientific Outputs
“Are Peer-Reviews of Grant Proposals Reliable? An Analysis of Economic and Social Research Council (ESRC) Funding Applications”, Jerrim & Vries 2020
“Is Scholarly Refereeing Productive (at the Margin)?”, Hadavand et al 2020
“The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases”, Schäfer & Schwarz 2019
“Registered Reports: an Early Example and Analysis”, Wiseman et al 2019
“The Sociology of Scientific Validity: How Professional Networks Shape Judgement in Peer Review”, Teplitskiy et al 2018
The sociology of scientific validity: How professional networks shape judgement in peer review
“Scientific Autonomy, Public Accountability, and the Rise of ‘Peer Review’ in the Cold War United States”, Baldwin 2018
“Low Agreement among Reviewers Evaluating the Same NIH Grant Applications”
Low agreement among reviewers evaluating the same NIH grant applications
“Uncertainty and Individual Discretion in Allocating Research Funds”, Goldstein & Kearney 2017
Uncertainty and Individual Discretion in Allocating Research Funds
“Can Results-Free Review Reduce Publication Bias? The Results and Implications of a Pilot Study”
Can Results-Free Review Reduce Publication Bias? The Results and Implications of a Pilot Study:
“To Apply or Not to Apply: A Survey Analysis of Grant Writing Costs and Benefits”, Hippel & Hippel 2015
To Apply or Not to Apply: A Survey Analysis of Grant Writing Costs and Benefits
“Protocol Review at The Lancet: 1997–2015”, Lancet 2015
“A Two-Step Manuscript Submission Process Can Reduce Publication Bias”, Smulders 2013
A two-step manuscript submission process can reduce publication bias:
“On the Time Spent Preparing Grant Proposals: an Observational Study of Australian Researchers”, Herbert et al 2013
On the time spent preparing grant proposals: an observational study of Australian researchers
“A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants”, Bornmann et al 2010
“Peer-Review in a World With Rational Scientists: Toward Selection of the Average”, Thurner & Hanel 2010
Peer-review in a world with rational scientists: Toward selection of the average
“Telescope Time Without Tears: A Distributed Approach to Peer Review”, Merrifield & Saari 2009
Telescope Time Without Tears: A Distributed Approach to Peer Review
“Science in the 21st Century: Social, Political, and Economic Issues”, González-Álvarez 2008
Science in the 21st century: social, political, and economic issues:
“Is Tenure Justified? An Experimental Study of Faculty Beliefs about Tenure, Promotion, and Academic Freedom”, Ceci et al 2007
“Effect of Blinded Peer Review on Abstract Acceptance”, Ross 2006
Effect of Blinded Peer Review on Abstract Acceptance:
View External Link:
“Effects of Editorial Peer Review: A Systematic Review”, Jefferson 2002
Effects of Editorial Peer Review: A Systematic Review:
View External Link:
“A Letter from the Frustrated Author of a Journal Paper”, Glass 2000
A letter from the frustrated author of a journal paper:
View PDF:
“Effect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports: A Randomized Controlled Trial”, Godlee 1998
View External Link:
“Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation”
Peer review for journals: Evidence on quality control, fairness, and innovation:
“Reflections After Refereeing Papers for NIPS”, Breiman 1995
“How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists”, Gans & Shepherd 1994
How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists
“The Influence of Prior Beliefs on Scientific Judgments of Evidence Quality”, Koehler 1993
The Influence of Prior Beliefs on Scientific Judgments of Evidence Quality
“Reviewer Bias”, Ernst et al 1992
“The Philosophical Basis of Peer Review and the Suppression of Innovation”, Horrobin 1990
The Philosophical Basis of Peer Review and the Suppression of Innovation
“The Story of the NIH Grants Programs”, Strickland 1989
“Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System”
Publication prejudices: An experimental study of confirmatory bias in the peer review system:
“Editorial [EJP Editorial on Registered Reports]”, Johnson 1975b
“Models of Control and Control of Bias”, Johnson 1975
“A Proposal for a New Editorial Policy in the Social Sciences”
A Proposal for a New Editorial Policy in the Social Sciences:
“Statistical Modeling: The Two Cultures”, Breiman 2024
“Real Peer Review Has Never Been Tried”
Sort By Magic
Annotations sorted by machine learning into inferred 'tags'. This provides an alternative way to browse: instead of by date order, one can browse in topic order. The 'sorted' list has been automatically clustered into multiple sections & auto-labeled for easier browsing.
Beginning with the newest annotation, it uses the embedding of each annotation to attempt to create a list of nearest-neighbor annotations, creating a progression of topics. For more details, see the link.
academic-beliefs
grant-writing
review-reliability
Miscellaneous
-
/doc/statistics/peer-review/1976-rosenthal-experimenterexpectancyeffects-ch3.pdf -
https://mattsclancy.substack.com/p/biases-against-risky-research -
https://mattsclancy.substack.com/p/can-taste-beat-peer-review -
https://mattsclancy.substack.com/p/what-does-peer-review-know -
https://theoryclass.wordpress.com/2013/06/06/a-mechanism-design-approach-to-peer-review/: -
https://www.cs.ucr.edu/~eamonn/Keogh_SIGKDD09_tutorial.pdf#page=8: -
https://www.experimental-history.com/p/the-rise-and-fall-of-peer-review -
https://www.noahpinion.blog/p/why-paul-ehrlich-got-everything-wrong -
https://www.statnews.com/2016/09/01/optogenetics/:View External Link:
Bibliography
-
https://arxiv.org/abs/2306.12587: “ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews”, -
https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-023-02882-y: “Saving Time and Money in Biomedical Publishing: the Case for Free-Format Submissions With Minimal Requirements”, -
https://journals.sagepub.com/doi/full/10.1177/25152459221128319: “Comparing Analysis Blinding With Preregistration in the Many-Analysts Religion Project”, -
https://journals.sagepub.com/doi/full/10.1177/25152459211007467: “An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports”, -
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0118494: “To Apply or Not to Apply: A Survey Analysis of Grant Writing Costs and Benefits”,