Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-745bb68f8f-b6zl4 Total loading time: 0 Render date: 2025-01-07T19:25:19.822Z Has data issue: false hasContentIssue false
Series:   Elements in Behavioural and Experimental Economics

Advances in Efficient Design of Experiments in Economics

Published online by Cambridge University Press:  04 December 2024

Michał Wiktor Krawczyk  and
John Masson Noble
Michał Wiktor Krawczyk
Affiliation:
University of Warsaw
John Masson Noble
Affiliation:
University of Warsaw

Summary

Amidst concerns about replicability but also thanks to the professionalisation of labs, the rise of pre-registration, the switch to online experiments, and enhanced computational power, experimental economics is undergoing rapid changes. They all call for efficient designs and data analysis, that is, they require that, given the constraints on participants' time, experiments provide as rich information as possible. In this Element the authors explore some ways in which this goal may be reached.
Get access

Keywords

methodology of experimentschoice experimentsadaptive designs in economicsapplied Bayesian statisticsoptimal designs
Type
Element
Information
Series: Elements in Behavioural and Experimental Economics
Online ISBN: 9781009263030
Publisher: Cambridge University Press
Print publication: 16 January 2025

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abdellaoui, Mohammed. Parameter-free elicitation of utility and probability weighting functions. Management Science, 46(11):1497–1512, 2000.CrossRefGoogle Scholar
Abdi, Hervé. Bonferroni and Šidák corrections for multiple comparisons. Encyclopedia of Measurement and Statistics, 3:103–107, 2007.Google Scholar
Agranov, Marina, Caplin, Andrew, and Tergiman, Chloe. Naive play and the process of choice in guessing games. Journal of the Economic Science Association, 1:146–157, 2015.CrossRefGoogle Scholar
Agresti, Alan. Categorical data analysis, volume 792. John Wiley & Sons, 2012.Google Scholar
Akaichi, Faical, Costa-Font, Joan, and Frank, Richard. Uninsured by choice? A choice experiment on long term care insurance. Journal of Economic Behavior & Organization, 173:422–434, 2020.CrossRefGoogle Scholar
Andreoni, James. Why free ride? Strategies and learning in public goods experiments. Journal of Public Economics, 37(3):291–304, 1988.CrossRefGoogle Scholar
Bleichrodt, Han, and Luis Pinto, Jose. A parameter-free elicitation of the probability weighting function in medical decision analysis. Management Science, 46(11):1485–1496, 2000.CrossRefGoogle Scholar
Box, E. P. George, Hunter, William Gordon, and Stuart Hunter, J.. Statistics for experimenters: Design, innovation, and discovery, 2nd edition. Wiley, 2005.Google Scholar
Brady, Henry E. Causation and explanation in social science. In Robert, E. Goodin, editor, The Oxford handbook of political science, Oxford University Press, 2008.Google Scholar
John, Bucknell, White, Justin S., and Shang, Ce. Can incentive-compatibility reduce hypothetical bias in smokers’ experimental choice behavior? A randomized discrete choice experiment. Journal of Choice Modelling, 37:100255, 2020.Google Scholar
Cavagnaro, Daniel R., Gonzalez, Richard, Myung, Jay I., and Pitt, Mark A.. Optimal decision stimuli for risky choice experiments: An adaptive approach. Management Science, 59(2):358–375, 2013.CrossRefGoogle ScholarPubMed
Cavagnaro, Daniel R., Myung, Jay I., Pitt, Mark A., and Kujala, Janne V.. Adaptive design optimization: A mutual information-based approach to model discrimination in cognitive science. Neural Computation, 22(4):887–905, 2010.CrossRefGoogle ScholarPubMed
Cetre, Sophie, Lobeck, Max, Senik, Claudia, and Verdier, Thierry. Preferences over income distribution: Evidence from a choice experiment. Journal of Economic Psychology, 74:102202, 2019.CrossRefGoogle Scholar
Chapman, Jonathan, Snowberg, Erik, Wang, Stephanie, and Camerer, Colin. Loss attitudes in the U.S. population: Evidence from Dynamically Optimized Sequential Experimentation (DOSE). Technical report, National Bureau of Economic Research, 2018.Google Scholar
Gary, Charness, Gneezy, Uri, and Kuhn, Michael A.. Experimental methods: Between-subject and within-subject design. Journal of Economic Behavior & Organization, 81(1):1–8, 2012.Google Scholar
Clark, Andrew E., Senik, Claudia, and Yamada, Katsunori. When experienced and decision utility concur: The case of income comparisons. Journal of Behavioral and Experimental Economics, 70:1–9, 2017.CrossRefGoogle Scholar
Croissant, Yves. Estimation of random utility models in r: The mlogit package. Journal of Statistical Software, 95(11):1–41, 2020. https://doi.org/10.18637/jss.v095.i11.CrossRefGoogle Scholar
Mikolaj, Czajkowski, Giergiczny, Marek, and Greene, William H.. Learning and fatigue effects revisited: Investigating the effects of accounting for unobservable preference and scale heterogeneity. Land Economics, 90(2):324–351, 2014.Google Scholar
Ding, Min. An incentive-aligned mechanism for conjoint analysis. Journal of Marketing Research, 44(2):214–223, 2007.CrossRefGoogle Scholar
Fox, Armando, Gribble, Steven D., Chawathe, Yatin, and Brewer, Eric A.. Adapting to network and client variation using infrastructural proxies: Lessons and perspectives. IEEE Personal Communications, 5(4):10–19, 1998.CrossRefGoogle Scholar
Frydman, Cary, Camerer, Colin, Bossaerts, Peter, and Rangel, Antonio. Maoa-l carriers are better at making optimal financial decisions under risk. Proceedings of the Royal Society B: Biological Sciences, 278(1714):2053–2059, 2011.CrossRefGoogle Scholar
Harrison, Glenn W., Lau, Morten I., and Elisabet Rutström, E.. Risk attitudes, randomization to treatment, and self-selection into experiments. Journal of Economic Behavior & Organization, 70(3):498–507, 2009.CrossRefGoogle Scholar
Hess, Stephane, and Palma, David. Apollo: A flexible, powerful and customisable freeware package for choice model estimation and application. Journal of Choice Modelling, 32:100170, 2019.CrossRefGoogle Scholar
Holland, Paul W. Causation and race. ETS Research Report Series, 2003(1):i–21, 2003.CrossRefGoogle Scholar
Holland, Paul W. Statistics and causal inference. Journal of the American Statistical Association, 81(396):945–960, 1986.CrossRefGoogle Scholar
Horiuchi, Yusaku, Markovich, Zachary, and Yamamoto, Teppei. Does conjoint analysis mitigate social desirability bias? Political Analysis, 30(4):535–549, 2022.CrossRefGoogle Scholar
Horowitz, John K. The Becker–DeGroot–Marschak mechanism is not necessarily incentive compatible, even for non-random goods. Economics Letters, 93(1):6–11, 2006.CrossRefGoogle Scholar
Jacquemet, Nicolas, and l’Haridon, Olivier. Experimental economics. Cambridge University Press, 2018.CrossRefGoogle Scholar
Cathleen, Johnson, Baillon, Aurélien, Bleichrodt, Han, Zhihua, Li, Van Dolder, Dennie, and Wakker, Peter P. PRINCE: An improved method for measuring incentivized preferences. Journal of Risk and Uncertainty, 62(1):1–28, 2021.Google Scholar
Kachurka, Raman, Krawczyk, Michał, and Rachubik, Joanna. State lottery in the lab: an experiment in external validity. Experimental Economics, 24: 1242–1266, 2021.CrossRefGoogle Scholar
Krawczyk, Michał. What should be regarded as deception in experimental economics? Evidence from a survey of researchers and subjects. Journal of Behavioral and Experimental Economics, 79:110–118, 2019.CrossRefGoogle Scholar
Krawczyk, Michał, Blasco, Andrea, Gajderowicz, Tomasz, and Giergiczny, Marek. Europeans’ attitudes towards displaced populations: Evidence from a conjoint experiment on support for temporary protection, 2023. Available at https://ssrn.com/abstract=4564737 or http://dx.doi.org/10.2139/ssrn.4564737.CrossRefGoogle Scholar
Krawczyk, Michał, and Sylwestrzak, Marta. Exploring the role of deliberation time in non-selfish behavior: The double response method. Journal of Behavioral and Experimental Economics, 72:121–134, 2018.CrossRefGoogle Scholar
Levitt, Steven D., and List, John A.. Field experiments in economics: The past, the present, and the future. European Economic Review, 53(1):1–18, 2009.CrossRefGoogle Scholar
Lindley, Dennis V. On a measure of the information provided by an experiment. The Annals of Mathematical Statistics, 27(4):986–1005, 1956.CrossRefGoogle Scholar
List, John. Sometimes winning means knowing when to quit. Wall Street Journal, 30 December 2021.Google Scholar
Mahoney, James, and Acosta, Laura. A regularity theory of causality for the social sciences. Quality & Quantity, 56:1889–1911, 2021.CrossRefGoogle Scholar
Mariel, Petr, Hoyos, David, Meyerhoff, Jürgen, Czajkowski, Mikolaj, Dekker, Thijs, Glenk, Klaus, Bredahl Jacobsen, Jette, Liebe, Ulf, Bøye Olsen, Søren, Sagebiel, Julian, and Thiene, Mara. Environmental valuation with discrete choice experiments: Guidance on design, implementation and data analysis. Springer Nature, 2021.CrossRefGoogle Scholar
Marks, David F., and Colwell, John. The psychic staring effect. Skeptical Inquirer, 24(5):41–49, 2000.Google Scholar
McFadden, Daniel. The measurement of urban travel demand. Journal of Public Economics, 3(4):303–328, 1974.CrossRefGoogle Scholar
McFadden, Daniel. Modeling the choice of residential location. In Karlqvist, Anders, Snickars, Folke, and Weibull, Jürgen, editors, Spatial interaction theory and planning models, pp. 75–96. North Holland, 1978.Google Scholar
Menapace, Luisa, and Raffaelli, Roberta. Unraveling hypothetical bias in discrete choice experiments. Journal of Economic Behavior & Organization, 176:416–430, 2020.CrossRefGoogle Scholar
Meyerhoff, Jürgen, Oehlmann, Malte, and Weller, Priska. The influence of design dimensions on stated choices in an environmental context. Environmental and Resource Economics, 61(3):385–407, 2015.CrossRefGoogle Scholar
Moffatt, Peter G. Experimetrics: Econometrics for experimental economics. Macmillan International Higher Education, 2015.Google Scholar
Overstall, Antony M., and Woods, David C.. Bayesian design of experiments using approximate coordinate exchange. Technometrics, 59(4):458–470, 2017.CrossRefGoogle Scholar
Papoutsi, Georgia Rodolfo, S. Nayga Jr, M., Lazaridis, Panagiotis, and Andreas, C. Drichoutis. Fat tax, subsidy or both? The role of information and children’s pester power in food choice. Journal of Economic Behavior & Organization, 117:196–208, 2015.CrossRefGoogle Scholar
Pearl, Judea. Causality. Cambridge University Press, 2009.CrossRefGoogle Scholar
Perny, Patrice, Viappiani, Paolo, and Boukhatem, Abdellah. Incremental preference elicitation for decision making under risk with the rank-dependent utility model. In Ihler, Alexander and Janzing, Dominik, editors, Uncertainty in artificial intelligence, pp. 597–606. AUAI Press for Association for Uncertainty in Artificial Intelligence, 2016.Google Scholar
Shannon, Claude Elwood. A mathematical theory of communication. The Bell System Technical Journal, 27(3):379–423, 1948.CrossRefGoogle Scholar
Shigeoka, Hitoshi, and Yamada, Katsunori. Income-comparison attitudes in the United States and the United Kingdom: Evidence from discrete-choice experiments. Journal of Economic Behavior & Organization, 164:414–438, 2019.CrossRefGoogle Scholar
Smith, Mike D. Biased coin randomization. In Balakrishnan, Narayanaswamy, editor, Methods and applications of statistics in clinical trials. Volume 1: Concepts, principles, trials, and design, pp. 90–105. Wiley, 2014.CrossRefGoogle Scholar
Sokol-Hessner, Peter, Hsu, Ming, Curley, Nina G., Delgado, Mauricio R., Camerer, Colin F., and Phelps, Elizabeth A.. Thinking like a trader selectively reduces individuals’ loss aversion. Proceedings of the National Academy of Sciences, 106(13):5035–5040, 2009.CrossRefGoogle Scholar
Spirtes, Peter, Glymour, Clark, and Scheines, Richard, with additional material by David Heckerman. Causation, prediction, and search. MIT Press, 2001.CrossRefGoogle Scholar
Steimle, Lauren N., Sun, Yuming, Johnson, Lauren, Besedeš, Tibor, Mokhtarian, Patricia, and Nazzal, Dima. Students’ preferences for returning to colleges and universities during the COVID-19 pandemic: A discrete choice experiment. Socio-economic Planning Sciences, page 101266, 2022.CrossRefGoogle Scholar
, Student. Appendix to Mercer and Hall’s paper ‘The experimental error of field trials’. Journal of Agricultural Science, 4, 128–131.Google Scholar
Vandana, Subroy, Rogers, Abbie A., and Kragt, Marit E.. To bait or not to bait: A discrete choice experiment on public preferences for native wildlife and conservation management in Western Australia. Ecological Economics, 147:114–122, 2018.Google Scholar
Thye, Shane. Logical and philosophical foundations of experimental research in the social sciences. In Webster, Murray and Sell, Jane, editors, Laboratory experiments in the social sciences, pp. 53–82. Elsevier, 2014.CrossRefGoogle Scholar
Titchener, Edward Bradford. Experimental psychology: A retrospect. American Journal of Psychology, 36(3):313–323, 1925.CrossRefGoogle Scholar
Toubia, Olivier, Hauser, John, and Garcia, Rosanna. Probabilistic polyhedral methods for adaptive choice-based conjoint analysis: Theory and application. Marketing Science, 26(5):596–610, 2007.CrossRefGoogle Scholar
Toubia, Olivier, Johnson, Eric, Evgeniou, Theodoros, and Delquié, Philippe. Dynamic experiments for estimating preferences: An adaptive method of eliciting time and risk parameters. Management Science, 59(3):613–640, 2013.CrossRefGoogle Scholar
Train, Kenneth E. Recreation demand models with taste differences over people. Land economics, University of Wisconsin Press, 74(2), 230–239, 1998.CrossRefGoogle Scholar
Tversky, Amos, and Kahneman, Daniel. Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5(4):297–323, 1992.CrossRefGoogle Scholar
Amos, Tversky, and Thaler, Richard H.. Anomalies: Preference reversals. Journal of Economic Perspectives, 4(2):201–211, 1990.Google Scholar
Vossler, Christian A., Doyon, Maurice, and Rondeau, Daniel. Truth in consequentiality: Theory and field evidence on discrete choice experiments. American Economic Journal: Microeconomics, 4(4):145–171, 2012.Google Scholar
Vossler, Christian A., and Evans, Mary F.. Bridging the gap between the field and the lab: Environmental goods, policy maker input, and consequentiality. Journal of Environmental Economics and Management, 58(3):338–345, 2009.CrossRefGoogle Scholar
Wakker, Peter, and Deneffe, Daniel. Eliciting von Neumann–Morgenstern utilities when probabilities are distorted or unknown. Management Science, 42(8):1131–1150, 1996.CrossRefGoogle Scholar
Woodward, James. Causation and manipulability. In Edward, N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Winter edition, 2016.Google Scholar
Ziliak, Stephen T. Field balanced versus randomized field experiments in economics: Why W. S. Gosset aka ‘Student’ matters. Review of Behavioral Economics, 1(1–2):167–208.CrossRefGoogle Scholar

Save element to Kindle

To save this element to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Advances in Efficient Design of Experiments in Economics
Available formats
×

Save element to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Advances in Efficient Design of Experiments in Economics
Available formats
×

Save element to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Advances in Efficient Design of Experiments in Economics
Available formats
×