Lessons for the peer review process
Our findings offer three lessons for improving the peer review process.
1. Shorter deadlines are extremely effective in improving the speed of the review process. Moreover, shorter deadlines generate little adverse effect on referees’ agreement rates, the quality of referee reports, or performance at other journals. Indeed, based on the results of the experiment, the Journal of Public Economics now uses a four-week deadline for all referees.
2. Cash incentives can generate significant improvements in review times and also increase referees’ willingness to submit reviews. However, it is important to pair cash incentives with reminders shortly before the deadline. Some journals, such as the American Economic Review, have been offering cash incentives without providing referees reminders about the incentives. In this situation, sending reminders would improve referee performance at little additional cost.
3. Social incentives can also improve referee performance, especially among subgroups such as tenured professors who are less responsive to deadlines and cash payments. Light social incentives, such as the Journal of Financial Economics’ policy of posting referee times by referee name, have small effects on review times. Stronger forms of social pressure – such as active management by editors during the review process in the form of personalised letters and reminders – could potentially be highly effective in improving efficiency.
More generally, our results reject the view that the review process in economics is much slower than in other fields, such as the natural sciences, purely because economics papers are more complex or difficult to review. Instead, our findings show that small changes in journals’ policies can substantially improve the peer review process at little cost.
Lessons for increasing prosocial behaviour
Beyond the peer review process, our results also offer some insights into the determinants of prosocial behaviour more broadly.
1. Attention matters – reminders and deadlines have significant impacts on behaviour. Nudges that bring the behaviour of interest to the top of individuals’ minds are a low-cost way to increase prosocial behaviour, consistent with a large literature in behavioural economics (Thaler and Sunstein 2008).
2. Monetary incentives can be effective in increasing some forms of prosocial behaviour. We find no evidence that intrinsic motivation is crowded out by financial incentives in the case of peer review, mirroring the results of Lacetera et al. (2013) in the case of blood donations. While crowd-out of intrinsic motivation could be larger in other settings, these results show that one should not dismiss corrective taxes or subsidies as a policy instrument simply because the behaviour one seeks to change has an important prosocial element.
3. Finally, social incentives can be effective even when other policy instruments are ineffective. This result echoes findings in other settings – such as voting (Gerber et al. 2008), campaign contributions (Perez-Truglia and Cruces 2013), and energy conservation (Allcott 2011) – and suggests that social incentives are a useful complement to price incentives and behavioural nudges.
References
Allcott, Hunt (2011), “Social Norms and Energy Conservation”, Journal of Public Economics, 95(9–10): 1082–1095.
Chetty, Raj, Emmanuel Saez, and László Sándor (2014), “How Can We Increase Prosocial Behavior? An Experiment with Referees at the Journal of Public Economics”, Journal of Economic Perspectives, 28(3).
Cruces, Guillermo, Ricardo Perez-Trugliad, and Martin Tetaza (2013), “Biased perceptions of income distribution and preferences for redistribution: Evidence from a survey experiment”, Journal of Public Economics, 98: 100–112.
DellaVigna, Stefano, John A List, and Ulrike Malmendier (2012), “Testing for Altruism and Social Pressure in Charitable Giving”, Quarterly Journal of Economics, 127(1): 1–56.
Ellison, Glenn (2002), “The Slowdown of the Economics Publishing Process”, Journal of Political Economy, 110(5): 947–993.
Gerber, Alan S, Donald P Green, and Cristopher W Larimer (2008), “Social Pressure and Voter Turnout: Evidence from a Large-scale Field Experiment”, American Political Science Review, 102(1): 33–48.
Holmstrom, Bengt and Paul Milgrom (1991), “Multitask Principal-agent Analyses: Incentives Contracts, Asset Ownership, and Job Design”, Journal of Law, Economics and Organization, 7: 24–52.
Lacetera, Nicola, Mario Macis, and Robert Slonim (2013), “Economic Rewards to Motivate Blood Donations”, Science, 340(6135): 927–928.
Ledyard, John O (1995), “Public Goods: A Survey of Experimental Research”, in John H Kagel and Alvin E Roth (eds.), Handbook of Experimental Economics, Princeton: Princeton University Press: 111–194.
Thaler, Richard H and Cass R Sunstein (2008), Nudge: Improving Decisions about Health, Wealth, and Happiness, New Haven: Yale University Press.
Titmuss, Richard M (1971), The Gift Relationship, London: George Allen and Unwin.
Vesterlund, Lise (2014), “Voluntary Giving to Public Goods: Moving Beyond the Linear VCM”, in John H Kagel and Alvin E Roth (eds.), Handbook of Experimental Economics, Vol 2, Princeton: Princeton University Press.
Footnotes
1 Similar social pressure interventions also have significant impacts on voting and charitable contributions (Gerber et al. 2008, DellaVigna et al. 2012).
2 The cash incentive increases the fraction of referees who agree to review a manuscript. The social incentive reduces agreement rates, while the shorter deadline has no impact. We find that the selection effects induced by these changes in agreement rates are modest and are unlikely to explain the observed changes in review times.
|