Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-8ctnn Total loading time: 0 Render date: 2024-12-25T05:08:16.286Z Has data issue: false hasContentIssue false
Series:   Elements in Experimental Political Science

Abstraction in Experimental Design

Testing the Tradeoffs

Published online by Cambridge University Press:  29 September 2022

Ryan Brutger ,
Joshua D. Kertzer ,
Jonathan Renshon  and
Chagai M. Weiss
Ryan Brutger
Affiliation:
University of California, Berkeley
Joshua D. Kertzer
Affiliation:
Harvard University, Massachusetts
Jonathan Renshon
Affiliation:
University of Wisconsin, Madison
Chagai M. Weiss
Affiliation:
University of Wisconsin, Madison

Summary

Political scientists designing experiments often face the question of how abstract or detailed their experimental stimuli should be. Typically, this question is framed in terms of tradeoffs relating to experimental control and generalizability: the more context introduced into studies, the less control, and the more difficulty generalizing the results. Yet, we have reason to question this tradeoff, and there is relatively little systematic evidence to rely on when calibrating the degree of abstraction in studies. We make two contributions. First, we provide a theoretical framework which identifies and considers the consequences of three dimensions of abstraction in experimental design: situational hypotheticality, actor identity, and contextual detail. Second, we field a range of survey experiments, varying these levels of abstraction. We find that situational hypotheticality does not substantively change experimental results, but increased contextual detail dampens treatment effects and the salience of actor identities moderates results in specific situations.
Get access

Keywords

experimentsabstractabstractiongeneralizabilitysurvey experiments
Type
Element
Information
Series: Elements in Experimental Political Science
Online ISBN: 9781108999533
Publisher: Cambridge University Press
Print publication: 27 October 2022

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adida, Claire L. 2015. “Do African Voters Favor Coethnics? Evidence from a Survey Experiment in Benin.” Journal of Experimental Political Science 2(1):1–11.Google Scholar
Aguinis, Herman, and Bradley, Kyle J.. 2014. “Best Practice Recommendations for Designing and Implementing Experimental Vignette Methodology Studies.” Organizational Research Methods 17(4):351–71.CrossRefGoogle Scholar
Alekseev, Aleksandr, Charness, Gary and Gneezy, Uri. 2017. “Experimental Methods: When and Why Contextual Instructions Are Important.” Journal of Economic Behavior & Organization 134:48–59.Google Scholar
Alexander, Cheryl S., and Jay Becker, Henry. 1978. “The Use of Vignettes in Survey Research.” Public Opinion Quarterly 42(1):93–104.Google Scholar
Althaus, Scott L. 1998. “Information Effects in Collective Preferences.” American Political Science Review 92(3):545–58.Google Scholar
Arceneaux, Kevin. 2012. “Cognitive Biases and the Strength of Political Arguments.” American Journal of Political Science 56(2):271–85.Google Scholar
Banerjee, Abhijit, Green, Donald P., McManus, Jeffery and Pande, Rohini. 2014. “Are Poor Voters Indifferent to Whether Elected Leaders are Criminal or Corrupt? A Vignette Experiment in Rural India.” Political Communication 31(3):391–407.CrossRefGoogle Scholar
Bansak, Kirk, Hainmueller, Jens, Hopkins, Daniel J. and Yamamoto, Teppei. 2021. “Beyond the Breaking Point? Survey Satisficing in Conjoint Experiments.” Political Science Research and Methods 9(1):53–71.Google Scholar
Barabas, Jason, and Jerit, Jennifer. 2010. “Are Survey Experiments Externally Valid?” American Political Science Review 104(2):226–42.Google Scholar
Bassan-Nygate, Lotem, and Weiss, Chagai M. 2022. “Party Competition and Cooperation Shape Affective Polarization: Evidence from Natural and Survey Experiments in Israel.” Comparative Political Studies 55(2):287–318.Google Scholar
Baum, Matthew A., and Groeling, Tim. 2009. “Shot by the Messenger: Partisan Cues and Public Opinion Regarding National Security and War.” Political Behavior 31(2):157–86.CrossRefGoogle Scholar
Bell, Mark S., and Quek, Kai. 2018. “Authoritarian Public Opinion and the Democratic Peace.” International Organization 72(1):227–42.Google Scholar
Berinsky, Adam J. 2009. In Time of War: Understanding American Public Opinion from World War II to Iraq. University of Chicago Press.Google Scholar
Berinsky, Adam J., Huber, Gregory A and Lenz, Gabriel S. 2012. “Evaluating Online Labor Markets for Experimental Research.” Political Analysis 20(3):351–68.Google Scholar
Marianne, Bertrand, and Mullainathan, Sendhil. 2004. “Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination.” American Economic Review 94(4):991–1013.Google Scholar
Boettcher, III, William, A. 2004. “The Prospects for Prospect Theory: An Empirical Evaluation of International Relations Applications of Framing and Loss Aversion.” Political Psychology 25(3):331–62.CrossRefGoogle Scholar
Boettcher III, William A., and Cobb, Michael D. 2006. “Echoes of Vietnam? Casualty Framing and Public Perceptions of Success and Failure in Iraq.” Journal of Conflict Resolution 50(6):831–54.Google Scholar
Bostyn, Dries H., Sevenhant, Sybren and Roets, Arne. 2018. “Of Mice, Men, and Trolleys: Hypothetical Judgment versus Real-Life Behavior in Trolley-Style Moral Dilemmas.” Psychological Science 29(7):1084–93.Google Scholar
Bracic, Ana, and Murdie, Amanda. 2020. “Human Rights Abused? Terrorist Labeling and Individual Reactions to Call to Action.” Political Research Quarterly 73(4):878–92.Google Scholar
Brader, Ted, Valentino, Nicholas A. and Suhay, Elizabeth. 2008. “What Triggers Public Opposition to Immigration? Anxiety, Group Cues, and Imigration.” American Journal of Political Science 52(4):959–78.CrossRefGoogle Scholar
Brooks, Deborah Jordan and Valentino, Benjamin A.. 2011. “A War of One’s Own: Understanding the Gender Gap in Support for War.” Public Opinion Quarterly 75(2):270–86.Google Scholar
Brutger, Ryan. 2021. “The Power of Compromise: Proposal Power, Partisanship, and Public Support in International Bargaining.” World Politics 73(1):128–66.Google Scholar
Brutger, Ryan, and Guisinger, Alexandra. 2021. “Labor Market Volatility, Gender, and Trade Preferences.” Journal of Experimental Political Science: 1–14. https://doi.org/10.1017/XPS.2021.9.Google Scholar
Brutger, Ryan, and Rathbun, Brian. 2021. “Fair Share?: Equality and Equity in American Attitudes towards Trade.” International Organization 75(3):880–900.Google Scholar
Brutger, Ryan, and Strezhnev, Anton. 2022. “International Investment Disputes, Media Coverage, and Backlash against International Law.” Journal of Conflict Resolution 66(6):983–1009.Google Scholar
Brutger, Ryan, Kertzer, Joshua D., Renshon, Jonathan, Tingley, Dustin and Weiss, Chagai M.. 2022. “Abstraction and Detail in Experimental Design.” American Journal of Political Science. Early View. https://doi.org/10.1111/ajps.12710.Google Scholar
Burden, Barry C., and Klofstad, Casey A.. 2005. “Affect and Cognition in Party Identification.” Political Psychology 26(6):869–86.Google Scholar
Burge, Camille, Wamble, Julian J. and Cuomo, Rachel. 2020. “A Certain Type of Descriptive Representative? Understanding How the Skin Tone and Gender of Candidates Influences Black Politics.” Journal of Politics 82(4):1596–601.CrossRefGoogle Scholar
Bush, Sarah Sunn, and Zetterberg, Pär. 2021. “Gender Quotas and International Reputation.” American Journal of Political Science 65(2):326–41.CrossRefGoogle Scholar
Butler, Daniel M., and Broockman, David E.. 2011. “Do Politicians Racially Discriminate Against Constituents? A Field Experiment on State Legislators.” American Journal of Political Science 55(3):436–77.CrossRefGoogle Scholar
Butler, Daniel M., Nickerson, David W. et al. 2011. “Can Learning Constituency Opinion Affect How Legislators Vote? Results from a Field Experiment.” Quarterly Journal of Political Science 6(1):55–83.Google Scholar
Butler, Daniel M., and Powell, Eleanor Neff. 2014. “Understanding the Party Brand: Experimental Evidence on the Role of Valence.” The Journal of Politics 76(2):492–505.CrossRefGoogle Scholar
Cacioppo, John T., and Petty, Richard E.. 1982. “The Need for Cognition.” Journal of Personality and Social Psychology 42(1):116–31.CrossRefGoogle Scholar
Camerer, Colin. 1997. “Rules for Experimenting in Psychology and Economics, and Why They Differ.” In Understanding Strategic Interaction ed. Albers, Wulf, Güth, Werner, Hammerstein, Peter, Moldovanu, Benny and Damme, Eric. Springer: 313–27.Google Scholar
Cantor, Nancy, and Mischel, Walter. 1979. “Prototypes in Person Perception.” Advances in Experimental Social Psychology 12:3–52.Google Scholar
Chandler, Jesse, Mueller, Pam and Paolacci, Gabriele. 2014. “Nonnaivete among Amazon Mechanical Turk Workers: Consequences and Solutions for Behavioral Researchers.” Behavioral Research Methods 46(1):112–30.CrossRefGoogle ScholarPubMed
Chapman, Terrence L., and Chaudoin, Stephen. 2020. “Public Reactions to International Legal Institutions: The International Criminal Court in a Developing Democracy.” The Journal of Politics 82(4):1305–20.Google Scholar
Chong, Dennis, and Druckman, James N.. 2007. “Framing Public Opinion in Competitive Democracies.” American Political Science Review 101(4):637–55.Google Scholar
Clarke, Kevin A., and Primo, David M.. 2012. A Model Discipline: Political Science and the Logic of Representations. Oxford University Press.Google Scholar
Clary, Christopher, and Siddiqui, Niloufer. 2021. “Voters and Foreign Policy: Evidence from a Conjoint Experiment in Pakistan.” Foreign Policy Analysis 17(2):1–13.CrossRefGoogle Scholar
Clifford, Scott, Sheagley, Geoffrey and Piston, Spencer. 2021. “Increasing Precision without Altering Treatment Effects: Repeated Measures Designs in Survey Experiments.” American Political Science Review 115(3):1048–65.Google Scholar
Colburn, Timothy, and Shute, Gary. 2007. “Abstraction in Computer Science.” Minds and Machines 17(2):169–84.Google Scholar
Colleau, Sophie M., Glynn, Kevin, Lybrand, Steven et al. 1990. “Symbolic Racism in Candidate Evaluation: An Experiment.” Political Behavior 12(4):385–402.Google Scholar
Collier, David, and Adcock, Robert. 2001. “Measurement Validity: A Shared Standard for Qualitative and Quantitative Research.” American Political Science Review 95(3):529–46.Google Scholar
Converse, Jean M., and Presser, Stanley. 1986. Survey Questions: Handcrafting the Standardized Questionnaire. SAGE Publications.Google Scholar
Coppock, Alexander. 2019. “Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach.” Political Science Research and Methods 7(3):613–28.Google Scholar
Coppock, Alexander, and McClellan, Oliver A.. 2019. “Validating the Demographic, Political, Psychological, and Experimental Results Obtained from a New Source of Online Survey Respondents.” Research & Politics 6(1):2053168018822174.Google Scholar
Croco, Sarah E., Hanmer, Michael J. and McDonald, Jared A.. 2021. “At What Cost? Reexamining Audience Costs in Realistic Settings.” Journal of Politics 83(1). https://doi.org/10.1086/708912.Google Scholar
Dafoe, Allan, Zhang, Baobao and Caughey, Devin. 2018. “Information Equivalence in Survey Experiments.” Political Analysis 26(4):399–416.Google Scholar
Dawes, Christopher T., Loewen, Peter John and Fowler, James H.. 2011. “Social Preferences and Political Participation.” The Journal of Politics 73(3):845–56.Google Scholar
Carpini, Delli, Michael, X., and Keeter, Scott. 1996. What Americans Know about Politics and Why It Matters. Yale University Press.Google Scholar
Dickson, Eric S. 2009. “Do Participants and Observers Assess Intentions Differently During Bargaining and Conflict?” American Journal of Political Science 53(4):910–30.Google Scholar
Dickson, Eric S. 2011. “Economics vs. Psychology Experiments: Stylization, Incentives, and Deception.” In Handbook of Experimental Political Science, ed. Druckman, James N., Green, Donald P., Kuklinski, James H. and Lupia, Arthur. Cambridge University Press.Google Scholar
Dill, Janina, and Schubiger, Livia I.. 2021. “Attitudes toward the Use of Force: Instrumental Imperatives, Moral Principles, and International Law.” American Journal of Political Science 65(3):612–33.CrossRefGoogle Scholar
Druckman, James N. 2003. “The Power of Television Images: The First Kennedy-Nixon Debate Revisited.” Journal of Politics 65(2):559–71.Google Scholar
Druckman, James N., and Nelson, Kjersten R.. 2003. “Framing and Deliberation: How Citizens’ Conversations Limit Elite Influence.” American Journal of Political Science 47(4):729–45.Google Scholar
Druckman, James N., and Kam, Cindy D.. 2011. “Students as Experimental Participants: A Defense of the ‘Narrow Data Base’.” In Handbook of Experimental Political Science, ed. Druckman, James N., Green, Donald P., Kuklinski, James H. and Lupia, Arthur. Cambridge University Press: 41–57.Google Scholar
Druckman, James N., Peterson, Erik and Slothuus, Rune. 2013. “How Elite Partisan Polarization Affects Public Opinion Formation.” American Political Science Review 107(1):57–79.Google Scholar
Dunning, Thad, and Harrison, Lauren. 2010. “Cross-cutting Cleavages and Ethnic Voting: An Experimental Study of Cousinage in Mali.” American Political Science Review 104(1):21–39.Google Scholar
Edwards, Pearce, and Arnon, Daniel. 2021. “Violence on Many Sides: Framing Effects on Protest and Support for Repression.” British Journal of Political Science 51(2):488–506.Google Scholar
Evers, Miles M., Fisher, Aleksandr and Schaaf, Steven D.. 2019. “Is There a Trump Effect? An Experiment on Political Polarization and Audience Costs.” Perspectives on Politics 17(2):433–52.Google Scholar
FeldmanHall, Oriel, Mobbs, Dean, Evans, Davy, et al. 2012. “What We Say and What We Do: The Relationship between Real and Hypothetical Moral Choices.” Cognition 123(3):434–41.Google Scholar
Findley, Michael G., Nielson, Daniel L. and Sharman, J. C.. 2013. “Using Field Experiments in International Relations: A Randomized Study of Anonymous Incorporation.” International Organization 67(4):657–93.CrossRefGoogle Scholar
Findley, Michael G., Kikuta, Kyosuke and Denly, Michael. 2021. “External Validity.” Annual Review of Political Science 24:365–93.Google Scholar
Fiske, Susan T., Lau, Richard R. and Smith, Richard A.. 1990. “On the Varieties and Utilities of Political Expertise.” Social Cognition 8(1):31–48.Google Scholar
Friedman, Sunder, Friedman, Daniel and Sunder, Shyam. 1994. Experimental Methods: A Primer for Economists. Cambridge University Press.Google Scholar
Gadarian, Shana Kushner, Goodman, Sara Wallace and Pepinsky, Thomas. 2021. “Partisan Endorsement Experiments Do Not Affect Mass Opinion on COVID-19.” Journal of Elections, Public Opinion and Parties 31(S1):122–31.Google Scholar
Gaddis, S. Michael. 2017. “How Black are Lakisha and Jamal? Racial Perceptions from Names Used in Correspondence Audit Studies.” Sociological Science 4:469–89.Google Scholar
Gaines, Brian J., Kuklinski, James H. and Quirk, Paul J.. 2007. “The Logic of the Survey Experiment Reexamined.” Political Analysis 15(1):1–20.Google Scholar
Green-Riley, Naima, Kruszewska-Eduardo, Dominika and Fu, Ze. 2021. “Teargas and Selfie Cams: Foreign Protests and Media in the Digital Age.” Journal of Experimental Political Science. First View. 1–1113. https://doi.org/10.1017/XPS.2021.1.Google Scholar
Guisinger, Alexandra. 2017. American Opinion on Trade: Preferences without Politics. Oxford University Press.Google Scholar
Haas, Nicholas, and Morton, Rebecca B. 2018. “Saying versus Doing: A New Donation Method for Measuring Ideal Points.” Public Choice 176(1):79–106.Google Scholar
Habyarimana, James, Humphreys, Macartan, Posner, Daniel N. and Weinstein, Jeremy M.. 2007. “Why Does Ethnic Diversity Undermine Public Goods Provision?” American Political Science Review 101(4):709–25.Google Scholar
Hainmueller, Jens, and Hopkins, Daniel J.. 2015. “The Hidden American Immigration Consensus: A Conjoint Analysis of Attitudes toward Immigrants.” American Journal of Political Science 59(3):529–48.Google Scholar
Hainmueller, Jens, Hangartner, Dominik and Yamamoto, Teppei. 2015. “Validating Vignette and Conjoint Survey Experiments against Real-World Behavior.” Proceedings of the National Academy of Sciences 112(8):2395–400.CrossRefGoogle ScholarPubMed
Hashtroudi, Shahin, Mutter, Sharon A., Cole, Elizabeth A. and Green, Susan K.. 1984. “Schema-Consistent and Schema-Inconsistent Information: Processing Demands.” Personality and Social Psychology Bulletin 10(2):269–78.Google Scholar
Hastie, Reid, and Park, Bernadette. 1986. “The Relationship between Memory and Judgment Depends on Whether the Judgment Task is Memory-Based or On-line.” Psychological review 93(3):258.Google Scholar
Herrmann, Richard K. 2013. “Perceptions and Image Theory in International Relations.” In Oxford Handbook of Political Psychology, ed. Huddy, Leonie, Sears, David O. and Levy, Jack S.. 2nd ed. Oxford University Press: 334–63.Google Scholar
Herrmann, Richard K., Tetlock, Philip E. and Visser, Penny S.. 1999. “Mass Public Decisions on Go to War: A Cognitive-Interactionist Framework.” American Political Science Review 93(3):553–73.Google Scholar
Hertwig, Ralph, and Ortmann, Andreas. 2008. “Deception in Experiments: Revisiting the Arguments in Its Defense.” Ethics & behavior 18(1):59–92.Google Scholar
Holland, Paul. 1986. “Statistics and Causal Inference.” Journal of the American Statistical Association 81(395):945–60.Google Scholar
Horowitz, Michael C., and Levendusky, Matthew S.. 2011. “Drafting Support for War: Conscription and Mass Support for Warfare.” Journal of Politics 73(2):524–34.CrossRefGoogle Scholar
Hsiao, Yuan, and Radnitz, Scott. 2021. “Allies or Agitators? How Partisan Identity Shapes Public Opinion about Violent or Nonviolent Protests.” Political Communication 38(4):479–97.Google Scholar
Huddleston, R. Joseph. 2019. “Think Ahead: Cost Discounting and External Validity in Foreign Policy Survey Experiments.” Journal of Experimental Political Science 6(2):108–19.Google Scholar
Huddy, Leonie, and Terkildsen, Nayda. 1993. “Gender Stereotypes and the Perception of Male and Female Candidates.” American Journal of Political Science 37(1):119–47.Google Scholar
Huff, Connor, and Kruszewska, Dominika. 2016. “Banners, Barricades, and Bombs: The Tactical Choices of Social Movements and Public Opinion.” Comparative Political Studies 49(13):1774–808.CrossRefGoogle Scholar
Huff, Connor, and Tingley, Dustin. 2015. ““Who Are These People?” Evaluating the Demographic Characteristics and Political Preferences of MTurk Survey Respondents.” Research & Politics 2(3). https://doi.org/10.1177/2053168015604648.Google Scholar
Hundley, Lindsay. 2020. “The Shadow of the Future and Bargaining Delay: An Experimental Approach.” Journal of Politics 82(1):378–83.Google Scholar
Irwin, Julie R., McClelland, Gary H. and Schulze, William D.. 1992. “Hypothetical and Real Consequences in Experimental Auctions for Insurance against Low-Probability Risks.” Journal of Behavioral Decision Making 5(2):107–16.Google Scholar
Johns, Robert, and Davies, Graeme A. M.. 2012. “Democratic Peace or Clash of Civilizations? Target States and Support for War in Britain and the United States.” Journal of Politics 74(4):1038–52.Google Scholar
Kam, Cindy D., and Zechmeister, Elizabeth J.. 2013. “Name Recognition and Candidate Support.” American Journal of Political Science 57(4):971–86.Google Scholar
Kanthak, Kristin, and Woon, Jonathan. 2015. “Women Don’t Run? Election Aversion and Candidate Entry.” American Journal of Political Science 59(3):595–612.Google Scholar
Kertzer, Joshua D. 2017. “Microfoundations in International Relations.” Conflict Management and Peace Science 34(1):81–97.Google Scholar
Kertzer, Joshua D. 2020. “Re-assessing Elite–Public Gaps in Political Behavior.” American Journal of Political Science. Early View. https://doi.org/10.1111/ajps.12583.Google Scholar
Kertzer, Joshua D., and Rathbun, Brian C.. 2015. “Fair Is Fair: Social Preferences and Reciprocity in International Politics.” World Politics 67(4):613–55.Google Scholar
Kertzer, Joshua D., and Brutger, Ryan. 2016. “Decomposing Audience Costs: Bringing the Audience Back into Audience Cost Theory.” American Journal of Political Science 60(1):234–49.Google Scholar
Kertzer, Joshua D., and Zeitzoff, Thomas. 2017. “A Bottom-Up Theory of Public Opinion about Foreign Policy.” American Journal of Political Science 61(3):543–58.Google Scholar
Kertzer, Joshua D., Renshon, Jonathan and Yarhi-Milo, Keren. 2021. “How Do Observers Assess Resolve?” British Journal of Political Science 51(1):308–30.Google Scholar
Kertzer, Joshua D., Brooks, Stephen G. and Jordan Brooks, Deborah. 2021. “Do Partisan Types Stop at the Water’s Edge?” Journal of Politics 83(4):1764–82.Google Scholar
Kertzer, Joshua D., and Renshon, Jonathan. 2022. “Elite Experiments and Surveys.” Annual Review of Political Science 25: 529–50.Google Scholar
Klar, Samara. 2018. “When Common Identities Decrease Trust: An Experimental Study of Partisan Women.” American Journal of Political Science 62(3):610–22.Google Scholar
Kreps, Sarah, and Roblin, Stephen. 2019. “Treatment Format and External Validity in International Relations Experiments.” International Interactions 45(3):576–94.Google Scholar
Kriner, Douglas L., and Shen, Francis X.. 2014. “Reassessing American Casualty Sensitivity: The Mediating Influence of Inequality.” Journal of Conflict Resolution 58(7):1174–201.Google Scholar
Landau-Wells, Marika. 2018. “Dealing with Danger: Threat Perception and Policy Preferences.” Doctoral Dissertation, Massachusetts Institute of Technology.Google Scholar
Landgrave, Michelangelo, and Weller, Nicholas. 2022. “Do Name-Based Treatments Violate Information Equivalence? Evidence from a Correspondence Audit Experiment.” Political Analysis 30(1):142–48.CrossRefGoogle Scholar
Lau, Richard R., Sigelman, Lee and Rovner, Ivy Brown. 2007. “The Effects of Negative Political Campaigns: A Meta-analytic Reassessment.” Journal of Politics 69(4):1176–209.Google Scholar
Lenzner, Timo, Kaczmirek, Lars and Lenzner, Alwine. 2010. “Cognitive Burden of Survey Questions and Response Times: A Psycholinguistic Experiment.” Applied Cognitive Psychology 24(7):1003–20.Google Scholar
LeVeck, Brad L., Alex Hughes, D., Fowler, James H., Hafner-Burton, Emilie M. and Victor, David G.. 2014. “The Role of Self-Interest in Elite Bargaining.” Proceedings of the National Academy of Sciences 111(52):18536–41.Google Scholar
LeVeck, Brad L., and Narang, Neil. 2017. “The Democratic Peace and the Wisdom of Crowds.” International Studies Quarterly 61(4):867–80.Google Scholar
Levine, David K., and Palfrey, Thomas R.. 2007. “The Paradox of Voter Participation? A Laboratory Study.” American Political Science Review 101(1):143–58.Google Scholar
Lupton, Danielle L. 2019. “The External Validity of College Student Subject Pools in Experimental Research: A Cross-Sample Comparison of Treatment Effect Heterogeneity.” Political Analysis 27(1):90–97.Google Scholar
Lyall, Jason, Blair, Graeme and Imai, Kosuke. 2013. “Explaining Support for Combatants during Wartime: A Survey Experiment in Afghanistan.” American Political Science Review 107(4):679–705.Google Scholar
Lyall, Jason, Zhou, Yang-Yang and Imai, Kosuke. 2020. “Can Economic Assistance Shape Combatant Support in Wartime? Experimental Evidence from Afghanistan.” American Political Science Review 114(1):126–43.Google Scholar
Mason, Lilliana. 2018. Uncivil Agreement: How Politics Became Our Identity. University of Chicago Press.Google Scholar
Matanock, Aila M., and Garbiras-Díaz, Natalia. 2018. “Considering Concessions: A Survey Experiment on the Colombian Peace Process.” Conflict Management and Peace Science 35(6):637–55.CrossRefGoogle Scholar
Matland, Richard E., and Tezcür, Güneş Murat. 2011. “Women as Candidates: An Experimental Study in Turkey.” Politics & Gender 7(3):365–90.Google Scholar
Mattes, Michaela, and Weeks, Jessica L. P.. 2019. “Hawks, Doves and Peace: An Experimental Approach.” American Journal of Political Science 63(1):53–66.Google Scholar
McDermott, Rose, and Cowden, Jonathan. 2001. “The Effects of Uncertainty and Sex in a Crisis Simulation Game.” International Interactions 27(4):353–80.Google Scholar
McDermott, Rose. 2002. “Experimental Methods in Political Science.” Annual Review of Political Science 5:31–61.CrossRefGoogle Scholar
McDermott, Rose, Johnson, Dominic, Cowden, Jonathan and Rosen, Stephen. 2007. “Testosterone and Aggression in a Simulated Crisis Game.” Annals of the American Academy of Political and Social Science 614(1):15–33.CrossRefGoogle Scholar
McDonald, Jared. 2020. “Avoiding the Hypothetical: Why ‘Mirror Experiments’ are an Essential Part of Survey Research.” International Journal of Public Opinion Research 32(2):266–83.Google Scholar
McDonald, Jared, Croco, Sarah E. and Turitto, Candace. 2019. “Teflon Don or Politics as Usual? An Examination of Foreign Policy Flip-Flops in the Age of Trump.” Journal of Politics 81(2):757–66.Google Scholar
Mendelberg, Tali. 2001. The Race Card: Campaign Strategy, Implicit Messages, and the Norm of Equality. Princeton University Press.Google Scholar
Mintz, Alex, and Geva, Nehemia. 1993. “Why Don’t Democracies Fight Each Other?: An Experimental Study.” Journal of Conflict Resolution 37(3):484–503.Google Scholar
Morton, Rebecca B., and Williams, Kenneth C. 2010. Experimental Political Science and the Study of Causality: From Nature to the Lab. Cambridge University Press.Google Scholar
Mullinix, Kevin J., Leeper, Thomas J., Druckman, James N. and Freese, Jeremy. 2015. “The Generalizability of Survey Experiments.” Journal of Experimental Political Science 2(2):109–38.Google Scholar
Mummolo, Jonathan, and Peterson, Erik. 2019. “Demand Effects in Survey Experiments: An Empirical Assessment.” American Political Science Review 113(2):517–29.Google Scholar
Mutz, Diana C. 2011. Population-Based Survey Experiments. Princeton University Press.Google Scholar
Mutz, Diana C. 2021. “Improving Experimental Treatments in Political Science.” In Advances in Experimental Political Science, ed. Druckman, James N. and Green, Donald P.. Cambridge University Press: 219–38.Google Scholar
Mutz, Diana C., and Kim, Eunji. 2017. “The Impact of In-Group Favoritism on Trade Preferences.” International Organization 71 (4):827–50.Google Scholar
Nelson, Thomas E., Clawson, Rosalee A. and Oxley, Zoe M.. 1997. “Media Framing of a Civil Liberties Conflict and Its Effect on Tolerance.” American Political Science Review 91(3):567–83.Google Scholar
Nicholson, Stephen P. 2012. “Polarizing Cues.” American Journal of Political Science 56(1):52–66.Google Scholar
Nielson, Daniel L., Hyde, Susan D. and Kelley, Judith. 2019. “The Elusive Sources of Legitimacy Beliefs: Civil Society Views of International Election Observers.” The Review of International Organizations 14(4):685–715.Google Scholar
Paivio, Allan. 1990. Mental Representations: A Dual Coding Approach. Oxford University Press.Google Scholar
Petsko, Christopher D., and Bodenhausen, Galen V.. 2019. “Racial Stereotyping of Gay Men: Can a Minority Sexual Orientation Erase Race?” Journal of Experimental Social Psychology 83:37–54.Google Scholar
Porter, Ethan, and Velez, Yamil R.. 2022. “Placebo Selection in Survey Experiments: An Agnostic Approach.” Political Analysis First View: 1–14. https://doi.org/10.1017/pan.2021.16.Google Scholar
Press, Daryl G., Sagan, Scott D. and Valentino, Benjamin A.. 2013. “Atomic Aversion: Experimental Evidence on Taboos, Traditions, and the Non-use of Nuclear Weapons.” American Political Science Review 107(1):188–206.Google Scholar
Quek, Kai. 2017. “Rationalist Experiments on War.” Political Science Research and Methods 5(1):123–42.Google Scholar
Raffler, P.J., 2020. Does political oversight of the bureaucracy increase accountability? Field experimental evidence from a dominant party regime. American Political Science Review. Early View. https://doi.org/10.1017/S0003055422000181.Google Scholar
Rathbun, Brian C., Kertzer, Joshua D. and Paradis, Mark. 2017. “Homo diplomaticus: Mixed-Method Evidence of Variation in Strategic Rationality 71:S1.” International Organization: S33–S60.CrossRefGoogle Scholar
Reeves, Andrew, and Rogowski, Jon C.. 2018. “The Public Cost of Unilateral Action.” American Journal of Political Science 62(2):424–40.Google Scholar
Reiley, David. 2015. “The Lab and the Field: Empirical and Experimental Economics.” In Handbook of Experimental Economic Methodology, ed. Fréchette, Guillaume R. and Schotter, Andrew. Oxford University Press: 410–12.Google Scholar
Renshon, Jonathan. 2015. “Losing Face and Sinking Costs: Experimental Evidence on the Judgment of Political and Military Leaders.” International Organization 69(3):659–95.Google Scholar
Renshon, Jonathan, Dafoe, Allan and Huth, Paul. 2018. “Leader Influence and Reputation Formation in World Politics.” American Journal of Political Science 62(2):325–39.Google Scholar
Rosenwasser, Shirley M., Rogers, Robyn R., Fling, Sheila, Silvers-Pickens, Kayla and Butemeyer, John. 1987. “Attitudes toward Women and Men in Politics: Perceived Male and Female Candidate Competencies and Participant Personality Characteristics.” Political Psychology 8(2):191–200.Google Scholar
Rousseau, David L., and Garcia-Retamero, Rocio. 2007. “Identity, Power, and Threat Perception: A Cross-national Experimental Study.” Journal of Conflict Resolution 51(5):744–71.Google Scholar
Rousu, Matthew C., Colson, Gregory, Corrigan, Jay R., Grebitus, Carola and Loureiro, Maria L.. 2015. “Deception in Experiments: Towards Guidelines on Use in Applied Economics Research.” Applied Economic Perspectives and Policy 37(3):524–36.Google Scholar
Rubenzer, Trevor, and Redd, Steven B.. 2010. “Ethnic Minority Groups and US Foreign Policy: Examining Congressional Decision Making and Economic Sanctions.” International Studies Quarterly 54(3):755–77.Google Scholar
Sanbonmatsu, Kira. 2002. “Gender Stereotypes and Vote Choice.” American Journal of Political Science 46(1):20–34.Google Scholar
Sartori, Giovanni. 1970. “Concept Misformation in Comparative Politics.” American Political Science Review 64(4):1033–53.Google Scholar
Saunders, Elizabeth N. 2018. “Leaders, Advisers, and the Political Origins of Elite Support for War.” Journal of Conflict Resolution 62(10):2118–49.Google Scholar
Schneider, Monica C. 2014. “The Effects of Gender-Bending on Candidate Evaluations.” Journal of Women, Politics & Policy 35(1):55–77.Google Scholar
Semin, Gün R., and Fiedler, Klaus. 1988. “The Cognitive Functions of Linguistic Categories in Describing Persons: Social Cognition and Language.” Journal of Personality and Social Psychology 54(4):558–68.Google Scholar
Shadish, William, Cook, Thomas D. and Thomas Campbell, Donald. 2002. Experimental and Quasi-experimental Designs for Generalized Causal Inference. Houghton Mifflin.Google Scholar
Shapira, Oren, Liberman, Nira, Trope, Yaacov and Rim, SoYon. 2012. “Levels of Mental Construal.” In SAGE Handbook of Social Cognition, ed. Fiske, Susan T. and Neil Macrae, C.. SAGE Publications: 229–50.Google Scholar
Smith, Vernon L. 1976. “Experimental Economics: Induced Value Theory.” The American Economic Review 66(2):274–79.Google Scholar
Sniderman, Paul M. 2011. “The Logic and Design of the Survey Experiment: An Autobiography of a Methodological Innovation.” In Handbook of Experimental Political Science, ed. Druckman, James N., Green, Donald P., Kuklinski, James H. and Lupia, Arthur. Cambridge University Press: 102–14.Google Scholar
Steiner, Peter M., Atzmüller, Christiane and Su, Dan. 2016. “Designing Valid and Reliable Vignette Experiments for Survey Research: A Case Study on the Fair Gender Income Gap.” Journal of Methods and Measurement in the Social Sciences 7(2):52–94.Google Scholar
Tankard, Margaret E., and Levy Paluck, Elizabeth. 2017. “The Effect of a Supreme Court Decision Regarding Gay Marriage on Social Norms and Personal Attitudes.” Psychological Science 28(9):1334–44.Google Scholar
Tannenwald, Nina. 1999. “The Nuclear Taboo: The United States and the Normative Basis of Nuclear Non-Use.” International Organization 53(3):433–68.Google Scholar
Teele, Dawn Langan, Kalla, Joshua and Rosenbluth, Frances. 2018. “The Ties That Double Bind: Social Roles and Women’s Underrepresentation in Politics.” American Political Science Review 112(3):525–41.Google Scholar
Tingley, Dustin. 2017. “Rising Power on the Mind.” International Organization 71(S1):S165–S188.Google Scholar
Tingley, Dustin, and Walter, Barbara. 2011a. “Reputation Building in International Relations: An Experimental Approach.” International Organization 65(2):343–65.Google Scholar
Tingley, Dustin, and Walter, Barbara. 2011b. “The Effect of Repeated Play on Reputation Building: An Experimental Approach.” International Organization 65(2):343–65.Google Scholar
Tomz, Michael. 2007. “Domestic Audience Costs in International Relations: An Experimental Approach.” International Organization 61(4):821–40.Google Scholar
Tomz, Michael, and Weeks, Jessica. 2013. “Public Opinion and the Democratic Peace.” American Political Science Review 107(4):849–65.CrossRefGoogle Scholar
Tomz, Michael, Weeks, Jessica and Yarhi-Milo, Keren. 2020. “Public Opinion and Decisions about Military Force in Democracies.” International Organization 74(1):119–43.Google Scholar
Trager, Robert F., and Vavreck, Lynn. 2011. “The Political Costs of Crisis Bargaining: Presidential Rhetoric and the Role of Party.” American Journal of Political Science 55(3):526–45.Google Scholar
Trope, Yaacov, and Liberman, Nira. 2003. “Temporal Construal.” Psychological Review 110(3):403–21.Google Scholar
Trope, Yaacov, and Liberman, Nira. 2010. “Construal-Level Theory of Psychological Distance.” Psychological Review 117(2):440–63.Google Scholar
Valentino, Nicholas A., Neuner, Fabian G. and Matthew Vandenbroek, L.. 2018. “The Changing Norms of Racial Political Rhetoric and the End of Racial Priming.” The Journal of Politics 80(3):757–71.Google Scholar
von Borzyskowski, Inken, and Vabulas, Felicity. 2021. “When Is It ‘OK’ to Leave?: U.S. Public Opinion toward Withdrawal from International Organizations.” Working paper.Google Scholar
Wamble, Julian J. 2020. “The Chosen One: How Community Commitment makes Certain Representatives More Preferable.” Working paper. https://www.julianwamble.com/_files/ugd/947e22_6a70c4f1b6c248a28020fdca4ded73d8.pdf.Google Scholar
White, Ariel, Strezhnev, Anton, Lucas, Christopher, Kruszewska, Dominika and Huff, Connor. 2018. “Investigator Characteristics and Respondent Behavior in Online Surveys.” Journal of Experimental Political Science 5(1):56–67.Google Scholar
White, Ariel R., Nathan, Noah L. and Faller, Julie K.. 2015. “What Do I Need to Vote? Bureaucratic Discretion and Discrimination by Local Election Officials.” American Political Science Review 109(1):129–42.Google Scholar
Xu, Sihua, Pan, Yu, Qu, Zhe et al. 2018. “Differential Effects of Real versus Hypothetical Monetary Reward Magnitude on Risk-Taking Behavior and Brain Activity.” Scientific Reports 8(1):1–9.Google ScholarPubMed
Yanow, Dvora, and Schwartz-Shea, Peregrine. 2016. “Encountering Your IRB 2.0: What Political Scientists Need to Know.” PS: Political Science & Politics 49(2):277–86.Google Scholar
Yarhi-Milo, Keren, Kertzer, Joshua D. and Renshon, Jonathan. 2018. “Tying Hands, Sinking Costs, and Leader Attributes.” Journal of Conflict Resolution 62(10):2150–79.Google Scholar

Save element to Kindle

To save this element to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Abstraction in Experimental Design
Available formats
×

Save element to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Abstraction in Experimental Design
Available formats
×

Save element to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Abstraction in Experimental Design
Available formats
×