Hostname: page-component-586b7cd67f-gb8f7 Total loading time: 0 Render date: 2024-11-30T15:10:00.936Z Has data issue: false hasContentIssue false
  • English
  • Français

Successfully scaled solutions need not be homogenous

Published online by Cambridge University Press:  09 July 2020

DILIP SOMAN  and
TANJIM HOSSAIN
DILIP SOMAN*
Affiliation:
Canada Research Chair in Behavioural Science and Economics at the Rotman School of Management, University of Toronto, Toronto, Canada
TANJIM HOSSAIN
Affiliation:
Professor of Marketing in the Department of Management at the University of Toronto Mississauga, Mississauga, Canada Rotman School of Management, University of Toronto, Toronto, Canada
*
*Correspondence to Rotman School of Management, University of Toronto, 105 St. George St., Toronto, ONM5S 3E6, Canada. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Al-Ubaydli et al. point out that many research findings experience a reduction in magnitude of treatment effects when scaled, and they make a number of proposals to improve the scalability of pilot project findings. While we agree that scalability is important for policy relevance, we argue that non-scalability does not always render a research finding useless in practice. Three practices ensuring (1) that the intervention is appropriate for the context; (2) that heterogeneity in treatment effects are understood; and (3) that the temptation to try multiple interventions simultaneously is avoided can allow us to customize successful policy prescriptions to specific real-world settings.

Type
Articles
Information
Behavioural Public Policy , Volume 5 , Special Issue 1: Field Experiments and Public Policy , January 2021 , pp. 80 - 89
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Al-Ubaydli, O., List, J. A. and Suskind, D. (2019), The science of using science: towards an understanding of the threats to scaling experiments. NBER Working Paper Series, No. 25848.10.3386/w25848CrossRefGoogle Scholar
Athey, S. and Imbens, G. (2016), ‘Recursive partitioning for heterogeneous causal effects’, Proceedings of the National Academy of Sciences of the United States of America, 113(27): 7353–7360. https://doi.org/10.1073/pnas.1510489113CrossRefGoogle ScholarPubMed
Bhargava, S. and Manoli, D. (2015), ‘Psychological frictions and the incomplete take-up of social benefits: evidence from an IRS field experiment’, American Economic Review, 105(11): 3489–3529.10.1257/aer.20121493CrossRefGoogle Scholar
Castelo, N., Hardy, E., House, J., Mazar, N., Tsai, C. and Zhao, M. (2015), ‘Moving citizens online: using salience & message framing to motivate behavior change’, Behavioral Science & Policy, 1(2): 69–81.10.1353/bsp.2015.0018CrossRefGoogle Scholar
Feinberg, R. A. (1986), ‘Credit cards as spending facilitating stimuli: a conditioning interpretation’, Journal of Consumer Research, 13(3): 348–356. doi: 10.1086/209074.CrossRefGoogle Scholar
Feng, B., Kim, M. and Soman, D. (in press), ‘Embedding behavioural insights in organizations’, Behaviorally informed organizations, Toronto: University of Toronto Press.Google Scholar
Fertig, A., Fishbane, A. and Lefkowiz, J. (2018), Using behavioral science to increase retirement savings in Mexico: a look at what we have learned over three years. Available at: http://www.ideas42.org/wp-content/uploads/2018/11/I42-1046_MetLifeLatAm_paper_ENG_Final.pdf.Google Scholar
Haynes, L., Service, O., Goldacre, B. and Torgerson, D. (2013), Test, learn, adapt: developing public policy with randomized controlled trials. Available at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/62529/TLA-1906126.pdf.10.2139/ssrn.2131581CrossRefGoogle Scholar
Hoffman, C. and Thommes, K. (2020), Using loss aversion to incentivize energy efficiency in a principal-agent context – evidence from a field experiment, forthcoming, Economics Letter.10.1016/j.econlet.2020.108984CrossRefGoogle Scholar
Hong, F., Hossain, T. and List, J. A. (2015), ‘Framing manipulations in contests: a natural field experiment’, Journal of Economic Behavior and Organization, 118, 372–382.10.1016/j.jebo.2015.02.014CrossRefGoogle Scholar
Hossain, T. and List, J. A. (2012), ‘The behavioralist visits the factory: increasing productivity using simple framing manipulations’, Management Science, 58(12): 2151–2167.10.1287/mnsc.1120.1544CrossRefGoogle Scholar
Karlan, D., McConnell, M., Mullainathan, S. and Zinman, J. (2016), ‘Getting to the top of mind: how reminders increase saving’, Management Science, 62(12): 3393–3411. https://doi.org/10.1287/mnsc.2015.2296CrossRefGoogle Scholar
Kilbourne, A. M., Neumann, M. S., Pincus, H. A., Bauer, M. S. and Stall, R. (2007), ‘Implementing evidence-based interventions in health care: application of the replicating effective programs framework’, Implementation Science, 2(42). doi:10.1186/1748-5908-2-42CrossRefGoogle ScholarPubMed
Kim, J. K., Yoon, J. H., Choi, Y. H. and Soman, D. (2020), Do text reminders about credit card spending help reduce spending? A quasi-experimental evaluation. Working paper.Google Scholar
Lee, J. S. and Yoon, Y. Y. (2011), Introduction of SMS transaction reminder (with cumulative spending). Available through: Financial Supervisory Service https://www.fss.or.kr.Google Scholar
Lichtenstein, S. and Slovic, P.. eds., (2006), The construction of preference, New York: Cambridge University Press.10.1017/CBO9780511618031CrossRefGoogle Scholar
OECD, (2017), Behavioural insights and public policy: lessons from around the world, Paris: OECD Publishing. doi:10.1787/9789264270480-en.Google Scholar
Prelec, D. and Simester, D. (2001), ‘Always leave home without it: a further investigation of the credit-card effect on willingness to pay’, Marketing Letters, 12(1): 5–12.10.1023/A:1008196717017CrossRefGoogle Scholar
Service, O., Hallsworth, M., Halpern, D., Algate, F., Gallagher, R., Nguyen, S., Ruda, S., Sanders, M. (2014), EAST: four simple ways to apply behavioural insights. Available at: https://www.bi.team/wp-content/uploads/2015/07/BIT-Publication-EAST_FA_WEB.pdf.Google Scholar
Shah, A., Osborne, M., Lefkowitz, J., Fishbane, A. and Soman, D. (2019), Can making family salient increase financial savings? Quantifying heterogeneous treatment effects in voluntary retirement contributions using a field experiment in Mexico. Available at SSRN: https://ssrn.com/abstract=3460722 or http://dx.doi.org/10.2139/ssrn.3460722.CrossRefGoogle Scholar
Shah, A., Osborne, M., Lefkowitz, J., Fishbane, A. and Soman, D. (2020), The simplification paradox: when reducing cognitive complexity can impede retirement savings contributions. Available at SSRN: https://ssrn.com/abstract=3460722 or http://dx.doi.org/10.2139/ssrn.3460722.CrossRefGoogle Scholar
Soman, D. (2001), ‘Effects of payment mechanism on spending behavior: the role of rehearsal and immediacy of payments’, Journal of Consumer Research, 27, 460–474.10.1086/319621CrossRefGoogle Scholar
Soman, D. (2015), The last mile: creating social and economic value from behavioural insights, Toronto: Rotman-UTP Publishing.Google Scholar
Soman, D. (2017), The elegant simplicity (and potential pitfalls) of simple frameworks, in OPRE Report 2017–23, Nudging Change in Human Services, Office of Planning, Research and Evaluation, U.S. Administration for Children and Families.Google Scholar
Soman, D. and Yeung, C. (in press), Preface, Behaviorally informed organizations, Toronto: University of Toronto Press.Google Scholar
Sparrow, B., Liu, J. and Wegner, D. M. (2011), ‘Google effects on memory: cognitive consequences of having information at our fingertips’, Science, 333(6043): 776–778.10.1126/science.1207745CrossRefGoogle ScholarPubMed
Thaler, R. H. (2015), Misbehaving: the making of behavioral economics, New York: W.W. Norton & Company.Google Scholar
Thaler, R. H. and Sunstein, C. R. (2008), Nudge: improving decisions about health, wealth, and happiness, Updated ed. New York, NY: Penguin Books.Google Scholar
Tversky, A. and Simonson, I. (1993), ‘Context-dependent preferences’, Management Science, 39(10): 1179–1189.10.1287/mnsc.39.10.1179CrossRefGoogle Scholar