Book contents
- Frontmatter
- Contents
- List of Figures
- List of Tables
- List of Boxes
- List of Contributors
- Acknowledgements
- 1 A New Era of Experimental Political Science
- Part I Experimental Designs
- Part II Experimental Data
- Part III Experimental Treatments and Measures
- Part IV Experimental Analys is and Presentation
- 14 Advances in Experimental Mediation Analysis
- 15 Subgroup Analysis: Pitfalls, Promise, and Honesty
- 16 Spillover Effects in Experimental Data
- 17 Visualize as You Randomize: Design-Based Statistical Graphs for Randomized Experiments
- Part V Experimental Reliability and Generalizability
- Part VI Using Experiments to study Identity
- Part VII Using Experiments to Study Government Actions
- Author Index
- Subject Index
15 - Subgroup Analysis: Pitfalls, Promise, and Honesty
from Part IV - Experimental Analys is and Presentation
Published online by Cambridge University Press: 08 March 2021
- Frontmatter
- Contents
- List of Figures
- List of Tables
- List of Boxes
- List of Contributors
- Acknowledgements
- 1 A New Era of Experimental Political Science
- Part I Experimental Designs
- Part II Experimental Data
- Part III Experimental Treatments and Measures
- Part IV Experimental Analys is and Presentation
- 14 Advances in Experimental Mediation Analysis
- 15 Subgroup Analysis: Pitfalls, Promise, and Honesty
- 16 Spillover Effects in Experimental Data
- 17 Visualize as You Randomize: Design-Based Statistical Graphs for Randomized Experiments
- Part V Experimental Reliability and Generalizability
- Part VI Using Experiments to study Identity
- Part VII Using Experiments to Study Government Actions
- Author Index
- Subject Index
Summary
Experiments often focus on recovering an average effect of a treatment on an outcome. A subgroup analysis involves identifying subgroups of observations for which the treatment is particularly efficacious or deleterious. Since these subgroups are not preregistered but instead discovered from the data, significant inferential issues emerge. We discuss methods for conduct honest inference on subgroups, meaning generating valid p-values and confidence intervals which account for the fact that the subgroups were not specified a priori. Central to this approach is the split-sample strategy, where half the data is used to identify effects and the other half to test them. After an intuitive and formal discussion of these issues, we provide simulation evidence and two examples illustrating these concepts in practice.
Keywords
- Type
- Chapter
- Information
- Advances in Experimental Political Science , pp. 271 - 288Publisher: Cambridge University PressPrint publication year: 2021
- 5
- Cited by