4 results
Replies to commentaries on beyond playing 20 questions with nature
- Abdullah Almaatouq, Thomas L. Griffiths, Jordan W. Suchow, Mark E. Whiting, James Evans, Duncan J. Watts
-
- Journal:
- Behavioral and Brain Sciences / Volume 47 / 2024
- Published online by Cambridge University Press:
- 05 February 2024, e65
-
- Article
- Export citation
-
Commentaries on the target article offer diverse perspectives on integrative experiment design. Our responses engage three themes: (1) Disputes of our characterization of the problem, (2) skepticism toward our proposed solution, and (3) endorsement of the solution, with accompanying discussions of its implementation in existing work and its potential for other domains. Collectively, the commentaries enhance our confidence in the promise and viability of integrative experiment design, while highlighting important considerations about how it is used.
Success stories cause false beliefs about success
- George Lifchits, Ashton Anderson, Daniel G. Goldstein, Jake M. Hofman, Duncan J. Watts
-
- Journal:
- Judgment and Decision Making / Volume 16 / Issue 6 / November 2021
- Published online by Cambridge University Press:
- 01 January 2023, pp. 1439-1463
-
- Article
-
- You have access Access
- Open access
- HTML
- Export citation
-
Many popular books and articles that purport to explain how people, companies, orideas succeed highlight a few successes chosen to fit a particular narrative. Weinvestigate what effect these highly selected “success narratives”have on readers’ beliefs and decisions. We conducted a large, randomized,pre-registered experiment, showing participants successful firms with foundersthat all either dropped out of or graduated college, and asked them to makeincentive-compatible bets on a new firm. Despite acknowledging biases in theexamples, participants’ decisions were very strongly influenced by them.People shown dropout founders were 55 percentage points more likely to bet on adropout-founded company than people who were shown graduate founders. Mostreported medium to high confidence in their bets, and many wrote causalexplanations justifying their decision. In light of recent concerns about falseinformation, our findings demonstrate how true but biased information canstrongly alter beliefs and decisions.
Beyond playing 20 questions with nature: Integrative experiment design in the social and behavioral sciences
- Abdullah Almaatouq, Thomas L. Griffiths, Jordan W. Suchow, Mark E. Whiting, James Evans, Duncan J. Watts
-
- Journal:
- Behavioral and Brain Sciences / Volume 47 / 2024
- Published online by Cambridge University Press:
- 21 December 2022, e33
-
- Article
- Export citation
-
The dominant paradigm of experiments in the social and behavioral sciences views an experiment as a test of a theory, where the theory is assumed to generalize beyond the experiment's specific conditions. According to this view, which Alan Newell once characterized as “playing twenty questions with nature,” theory is advanced one experiment at a time, and the integration of disparate findings is assumed to happen via the scientific publishing process. In this article, we argue that the process of integration is at best inefficient, and at worst it does not, in fact, occur. We further show that the challenge of integration cannot be adequately addressed by recently proposed reforms that focus on the reliability and replicability of individual findings, nor simply by conducting more or larger experiments. Rather, the problem arises from the imprecise nature of social and behavioral theories and, consequently, a lack of commensurability across experiments conducted under different conditions. Therefore, researchers must fundamentally rethink how they design experiments and how the experiments relate to theory. We specifically describe an alternative framework, integrative experiment design, which intrinsically promotes commensurability and continuous integration of knowledge. In this paradigm, researchers explicitly map the design space of possible experiments associated with a given research question, embracing many potentially relevant theories rather than focusing on just one. Researchers then iteratively generate theories and test them with experiments explicitly sampled from the design space, allowing results to be integrated across experiments. Given recent methodological and technological developments, we conclude that this approach is feasible and would generate more-reliable, more-cumulative empirical and theoretical knowledge than the current paradigm – and with far greater efficiency.
Contributors
-
- By Rose Teteki Abbey, K. C. Abraham, David Tuesday Adamo, LeRoy H. Aden, Efrain Agosto, Victor Aguilan, Gillian T. W. Ahlgren, Charanjit Kaur AjitSingh, Dorothy B E A Akoto, Giuseppe Alberigo, Daniel E. Albrecht, Ruth Albrecht, Daniel O. Aleshire, Urs Altermatt, Anand Amaladass, Michael Amaladoss, James N. Amanze, Lesley G. Anderson, Thomas C. Anderson, Victor Anderson, Hope S. Antone, María Pilar Aquino, Paula Arai, Victorio Araya Guillén, S. Wesley Ariarajah, Ellen T. Armour, Brett Gregory Armstrong, Atsuhiro Asano, Naim Stifan Ateek, Mahmoud Ayoub, John Alembillah Azumah, Mercedes L. García Bachmann, Irena Backus, J. Wayne Baker, Mieke Bal, Lewis V. Baldwin, William Barbieri, António Barbosa da Silva, David Basinger, Bolaji Olukemi Bateye, Oswald Bayer, Daniel H. Bays, Rosalie Beck, Nancy Elizabeth Bedford, Guy-Thomas Bedouelle, Chorbishop Seely Beggiani, Wolfgang Behringer, Christopher M. Bellitto, Byard Bennett, Harold V. Bennett, Teresa Berger, Miguel A. Bernad, Henley Bernard, Alan E. Bernstein, Jon L. Berquist, Johannes Beutler, Ana María Bidegain, Matthew P. Binkewicz, Jennifer Bird, Joseph Blenkinsopp, Dmytro Bondarenko, Paulo Bonfatti, Riet en Pim Bons-Storm, Jessica A. Boon, Marcus J. Borg, Mark Bosco, Peter C. Bouteneff, François Bovon, William D. Bowman, Paul S. Boyer, David Brakke, Richard E. Brantley, Marcus Braybrooke, Ian Breward, Ênio José da Costa Brito, Jewel Spears Brooker, Johannes Brosseder, Nicholas Canfield Read Brown, Robert F. Brown, Pamela K. Brubaker, Walter Brueggemann, Bishop Colin O. Buchanan, Stanley M. Burgess, Amy Nelson Burnett, J. Patout Burns, David B. Burrell, David Buttrick, James P. Byrd, Lavinia Byrne, Gerado Caetano, Marcos Caldas, Alkiviadis Calivas, William J. Callahan, Salvatore Calomino, Euan K. Cameron, William S. Campbell, Marcelo Ayres Camurça, Daniel F. Caner, Paul E. Capetz, Carlos F. Cardoza-Orlandi, Patrick W. Carey, Barbara Carvill, Hal Cauthron, Subhadra Mitra Channa, Mark D. Chapman, James H. Charlesworth, Kenneth R. Chase, Chen Zemin, Luciano Chianeque, Philip Chia Phin Yin, Francisca H. Chimhanda, Daniel Chiquete, John T. Chirban, Soobin Choi, Robert Choquette, Mita Choudhury, Gerald Christianson, John Chryssavgis, Sejong Chun, Esther Chung-Kim, Charles M. A. Clark, Elizabeth A. Clark, Sathianathan Clarke, Fred Cloud, John B. Cobb, W. Owen Cole, John A Coleman, John J. Collins, Sylvia Collins-Mayo, Paul K. Conkin, Beth A. Conklin, Sean Connolly, Demetrios J. Constantelos, Michael A. Conway, Paula M. Cooey, Austin Cooper, Michael L. Cooper-White, Pamela Cooper-White, L. William Countryman, Sérgio Coutinho, Pamela Couture, Shannon Craigo-Snell, James L. Crenshaw, David Crowner, Humberto Horacio Cucchetti, Lawrence S. Cunningham, Elizabeth Mason Currier, Emmanuel Cutrone, Mary L. Daniel, David D. Daniels, Robert Darden, Rolf Darge, Isaiah Dau, Jeffry C. Davis, Jane Dawson, Valentin Dedji, John W. de Gruchy, Paul DeHart, Wendy J. Deichmann Edwards, Miguel A. De La Torre, George E. Demacopoulos, Thomas de Mayo, Leah DeVun, Beatriz de Vasconcellos Dias, Dennis C. Dickerson, John M. Dillon, Luis Miguel Donatello, Igor Dorfmann-Lazarev, Susanna Drake, Jonathan A. Draper, N. Dreher Martin, Otto Dreydoppel, Angelyn Dries, A. J. Droge, Francis X. D'Sa, Marilyn Dunn, Nicole Wilkinson Duran, Rifaat Ebied, Mark J. Edwards, William H. Edwards, Leonard H. Ehrlich, Nancy L. Eiesland, Martin Elbel, J. Harold Ellens, Stephen Ellingson, Marvin M. Ellison, Robert Ellsberg, Jean Bethke Elshtain, Eldon Jay Epp, Peter C. Erb, Tassilo Erhardt, Maria Erling, Noel Leo Erskine, Gillian R. Evans, Virginia Fabella, Michael A. Fahey, Edward Farley, Margaret A. Farley, Wendy Farley, Robert Fastiggi, Seena Fazel, Duncan S. Ferguson, Helwar Figueroa, Paul Corby Finney, Kyriaki Karidoyanes FitzGerald, Thomas E. FitzGerald, John R. Fitzmier, Marie Therese Flanagan, Sabina Flanagan, Claude Flipo, Ronald B. Flowers, Carole Fontaine, David Ford, Mary Ford, Stephanie A. Ford, Jim Forest, William Franke, Robert M. Franklin, Ruth Franzén, Edward H. Friedman, Samuel Frouisou, Lorelei F. Fuchs, Jojo M. Fung, Inger Furseth, Richard R. Gaillardetz, Brandon Gallaher, China Galland, Mark Galli, Ismael García, Tharscisse Gatwa, Jean-Marie Gaudeul, Luis María Gavilanes del Castillo, Pavel L. Gavrilyuk, Volney P. Gay, Metropolitan Athanasios Geevargis, Kondothra M. George, Mary Gerhart, Simon Gikandi, Maurice Gilbert, Michael J. Gillgannon, Verónica Giménez Beliveau, Terryl Givens, Beth Glazier-McDonald, Philip Gleason, Menghun Goh, Brian Golding, Bishop Hilario M. Gomez, Michelle A. Gonzalez, Donald K. Gorrell, Roy Gottfried, Tamara Grdzelidze, Joel B. Green, Niels Henrik Gregersen, Cristina Grenholm, Herbert Griffiths, Eric W. Gritsch, Erich S. Gruen, Christoffer H. Grundmann, Paul H. Gundani, Jon P. Gunnemann, Petre Guran, Vidar L. Haanes, Jeremiah M. Hackett, Getatchew Haile, Douglas John Hall, Nicholas Hammond, Daphne Hampson, Jehu J. Hanciles, Barry Hankins, Jennifer Haraguchi, Stanley S. Harakas, Anthony John Harding, Conrad L. Harkins, J. William Harmless, Marjory Harper, Amir Harrak, Joel F. Harrington, Mark W. Harris, Susan Ashbrook Harvey, Van A. Harvey, R. Chris Hassel, Jione Havea, Daniel Hawk, Diana L. Hayes, Leslie Hayes, Priscilla Hayner, S. Mark Heim, Simo Heininen, Richard P. Heitzenrater, Eila Helander, David Hempton, Scott H. Hendrix, Jan-Olav Henriksen, Gina Hens-Piazza, Carter Heyward, Nicholas J. Higham, David Hilliard, Norman A. Hjelm, Peter C. Hodgson, Arthur Holder, M. Jan Holton, Dwight N. Hopkins, Ronnie Po-chia Hsia, Po-Ho Huang, James Hudnut-Beumler, Jennifer S. Hughes, Leonard M. Hummel, Mary E. Hunt, Laennec Hurbon, Mark Hutchinson, Susan E. Hylen, Mary Beth Ingham, H. Larry Ingle, Dale T. Irvin, Jon Isaak, Paul John Isaak, Ada María Isasi-Díaz, Hans Raun Iversen, Margaret C. Jacob, Arthur James, Maria Jansdotter-Samuelsson, David Jasper, Werner G. Jeanrond, Renée Jeffery, David Lyle Jeffrey, Theodore W. Jennings, David H. Jensen, Robin Margaret Jensen, David Jobling, Dale A. Johnson, Elizabeth A. Johnson, Maxwell E. Johnson, Sarah Johnson, Mark D. Johnston, F. Stanley Jones, James William Jones, John R. Jones, Alissa Jones Nelson, Inge Jonsson, Jan Joosten, Elizabeth Judd, Mulambya Peggy Kabonde, Robert Kaggwa, Sylvester Kahakwa, Isaac Kalimi, Ogbu U. Kalu, Eunice Kamaara, Wayne C. Kannaday, Musimbi Kanyoro, Veli-Matti Kärkkäinen, Frank Kaufmann, Léon Nguapitshi Kayongo, Richard Kearney, Alice A. Keefe, Ralph Keen, Catherine Keller, Anthony J. Kelly, Karen Kennelly, Kathi Lynn Kern, Fergus Kerr, Edward Kessler, George Kilcourse, Heup Young Kim, Kim Sung-Hae, Kim Yong-Bock, Kim Yung Suk, Richard King, Thomas M. King, Robert M. Kingdon, Ross Kinsler, Hans G. Kippenberg, Cheryl A. Kirk-Duggan, Clifton Kirkpatrick, Leonid Kishkovsky, Nadieszda Kizenko, Jeffrey Klaiber, Hans-Josef Klauck, Sidney Knight, Samuel Kobia, Robert Kolb, Karla Ann Koll, Heikki Kotila, Donald Kraybill, Philip D. W. Krey, Yves Krumenacker, Jeffrey Kah-Jin Kuan, Simanga R. Kumalo, Peter Kuzmic, Simon Shui-Man Kwan, Kwok Pui-lan, André LaCocque, Stephen E. Lahey, John Tsz Pang Lai, Emiel Lamberts, Armando Lampe, Craig Lampe, Beverly J. Lanzetta, Eve LaPlante, Lizette Larson-Miller, Ariel Bybee Laughton, Leonard Lawlor, Bentley Layton, Robin A. Leaver, Karen Lebacqz, Archie Chi Chung Lee, Marilyn J. Legge, Hervé LeGrand, D. L. LeMahieu, Raymond Lemieux, Bill J. Leonard, Ellen M. Leonard, Outi Leppä, Jean Lesaulnier, Nantawan Boonprasat Lewis, Henrietta Leyser, Alexei Lidov, Bernard Lightman, Paul Chang-Ha Lim, Carter Lindberg, Mark R. Lindsay, James R. Linville, James C. Livingston, Ann Loades, David Loades, Jean-Claude Loba-Mkole, Lo Lung Kwong, Wati Longchar, Eleazar López, David W. Lotz, Andrew Louth, Robin W. Lovin, William Luis, Frank D. Macchia, Diarmaid N. J. MacCulloch, Kirk R. MacGregor, Marjory A. MacLean, Donald MacLeod, Tomas S. Maddela, Inge Mager, Laurenti Magesa, David G. Maillu, Fortunato Mallimaci, Philip Mamalakis, Kä Mana, Ukachukwu Chris Manus, Herbert Robinson Marbury, Reuel Norman Marigza, Jacqueline Mariña, Antti Marjanen, Luiz C. L. Marques, Madipoane Masenya (ngwan'a Mphahlele), Caleb J. D. Maskell, Steve Mason, Thomas Massaro, Fernando Matamoros Ponce, András Máté-Tóth, Odair Pedroso Mateus, Dinis Matsolo, Fumitaka Matsuoka, John D'Arcy May, Yelena Mazour-Matusevich, Theodore Mbazumutima, John S. McClure, Christian McConnell, Lee Martin McDonald, Gary B. McGee, Thomas McGowan, Alister E. McGrath, Richard J. McGregor, John A. McGuckin, Maud Burnett McInerney, Elsie Anne McKee, Mary B. McKinley, James F. McMillan, Ernan McMullin, Kathleen E. McVey, M. Douglas Meeks, Monica Jyotsna Melanchthon, Ilie Melniciuc-Puica, Everett Mendoza, Raymond A. Mentzer, William W. Menzies, Ina Merdjanova, Franziska Metzger, Constant J. Mews, Marvin Meyer, Carol Meyers, Vasile Mihoc, Gunner Bjerg Mikkelsen, Maria Inêz de Castro Millen, Clyde Lee Miller, Bonnie J. Miller-McLemore, Alexander Mirkovic, Paul Misner, Nozomu Miyahira, R. W. L. Moberly, Gerald Moede, Aloo Osotsi Mojola, Sunanda Mongia, Rebeca Montemayor, James Moore, Roger E. Moore, Craig E. Morrison O.Carm, Jeffry H. Morrison, Keith Morrison, Wilson J. Moses, Tefetso Henry Mothibe, Mokgethi Motlhabi, Fulata Moyo, Henry Mugabe, Jesse Ndwiga Kanyua Mugambi, Peggy Mulambya-Kabonde, Robert Bruce Mullin, Pamela Mullins Reaves, Saskia Murk Jansen, Heleen L. Murre-Van den Berg, Augustine Musopole, Isaac M. T. Mwase, Philomena Mwaura, Cecilia Nahnfeldt, Anne Nasimiyu Wasike, Carmiña Navia Velasco, Thulani Ndlazi, Alexander Negrov, James B. Nelson, David G. Newcombe, Carol Newsom, Helen J. Nicholson, George W. E. Nickelsburg, Tatyana Nikolskaya, Damayanthi M. A. Niles, Bertil Nilsson, Nyambura Njoroge, Fidelis Nkomazana, Mary Beth Norton, Christian Nottmeier, Sonene Nyawo, Anthère Nzabatsinda, Edward T. Oakes, Gerald O'Collins, Daniel O'Connell, David W. Odell-Scott, Mercy Amba Oduyoye, Kathleen O'Grady, Oyeronke Olajubu, Thomas O'Loughlin, Dennis T. Olson, J. Steven O'Malley, Cephas N. Omenyo, Muriel Orevillo-Montenegro, César Augusto Ornellas Ramos, Agbonkhianmeghe E. Orobator, Kenan B. Osborne, Carolyn Osiek, Javier Otaola Montagne, Douglas F. Ottati, Anna May Say Pa, Irina Paert, Jerry G. Pankhurst, Aristotle Papanikolaou, Samuele F. Pardini, Stefano Parenti, Peter Paris, Sung Bae Park, Cristián G. Parker, Raquel Pastor, Joseph Pathrapankal, Daniel Patte, W. Brown Patterson, Clive Pearson, Keith F. Pecklers, Nancy Cardoso Pereira, David Horace Perkins, Pheme Perkins, Edward N. Peters, Rebecca Todd Peters, Bishop Yeznik Petrossian, Raymond Pfister, Peter C. Phan, Isabel Apawo Phiri, William S. F. Pickering, Derrick G. Pitard, William Elvis Plata, Zlatko Plese, John Plummer, James Newton Poling, Ronald Popivchak, Andrew Porter, Ute Possekel, James M. Powell, Enos Das Pradhan, Devadasan Premnath, Jaime Adrían Prieto Valladares, Anne Primavesi, Randall Prior, María Alicia Puente Lutteroth, Eduardo Guzmão Quadros, Albert Rabil, Laurent William Ramambason, Apolonio M. Ranche, Vololona Randriamanantena Andriamitandrina, Lawrence R. Rast, Paul L. Redditt, Adele Reinhartz, Rolf Rendtorff, Pål Repstad, James N. Rhodes, John K. Riches, Joerg Rieger, Sharon H. Ringe, Sandra Rios, Tyler Roberts, David M. Robinson, James M. Robinson, Joanne Maguire Robinson, Richard A. H. Robinson, Roy R. Robson, Jack B. Rogers, Maria Roginska, Sidney Rooy, Rev. Garnett Roper, Maria José Fontelas Rosado-Nunes, Andrew C. Ross, Stefan Rossbach, François Rossier, John D. Roth, John K. Roth, Phillip Rothwell, Richard E. Rubenstein, Rosemary Radford Ruether, Markku Ruotsila, John E. Rybolt, Risto Saarinen, John Saillant, Juan Sanchez, Wagner Lopes Sanchez, Hugo N. Santos, Gerhard Sauter, Gloria L. Schaab, Sandra M. Schneiders, Quentin J. Schultze, Fernando F. Segovia, Turid Karlsen Seim, Carsten Selch Jensen, Alan P. F. Sell, Frank C. Senn, Kent Davis Sensenig, Damían Setton, Bal Krishna Sharma, Carolyn J. Sharp, Thomas Sheehan, N. Gerald Shenk, Christian Sheppard, Charles Sherlock, Tabona Shoko, Walter B. Shurden, Marguerite Shuster, B. Mark Sietsema, Batara Sihombing, Neil Silberman, Clodomiro Siller, Samuel Silva-Gotay, Heikki Silvet, John K. Simmons, Hagith Sivan, James C. Skedros, Abraham Smith, Ashley A. Smith, Ted A. Smith, Daud Soesilo, Pia Søltoft, Choan-Seng (C. S.) Song, Kathryn Spink, Bryan Spinks, Eric O. Springsted, Nicolas Standaert, Brian Stanley, Glen H. Stassen, Karel Steenbrink, Stephen J. Stein, Andrea Sterk, Gregory E. Sterling, Columba Stewart, Jacques Stewart, Robert B. Stewart, Cynthia Stokes Brown, Ken Stone, Anne Stott, Elizabeth Stuart, Monya Stubbs, Marjorie Hewitt Suchocki, David Kwang-sun Suh, Scott W. Sunquist, Keith Suter, Douglas Sweeney, Charles H. Talbert, Shawqi N. Talia, Elsa Tamez, Joseph B. Tamney, Jonathan Y. Tan, Yak-Hwee Tan, Kathryn Tanner, Feiya Tao, Elizabeth S. Tapia, Aquiline Tarimo, Claire Taylor, Mark Lewis Taylor, Bishop Abba Samuel Wolde Tekestebirhan, Eugene TeSelle, M. Thomas Thangaraj, David R. Thomas, Andrew Thornley, Scott Thumma, Marcelo Timotheo da Costa, George E. “Tink” Tinker, Ola Tjørhom, Karen Jo Torjesen, Iain R. Torrance, Fernando Torres-Londoño, Archbishop Demetrios [Trakatellis], Marit Trelstad, Christine Trevett, Phyllis Trible, Johannes Tromp, Paul Turner, Robert G. Tuttle, Archbishop Desmond Tutu, Peter Tyler, Anders Tyrberg, Justin Ukpong, Javier Ulloa, Camillus Umoh, Kristi Upson-Saia, Martina Urban, Monica Uribe, Elochukwu Eugene Uzukwu, Richard Vaggione, Gabriel Vahanian, Paul Valliere, T. J. Van Bavel, Steven Vanderputten, Peter Van der Veer, Huub Van de Sandt, Louis Van Tongeren, Luke A. Veronis, Noel Villalba, Ramón Vinke, Tim Vivian, David Voas, Elena Volkova, Katharina von Kellenbach, Elina Vuola, Timothy Wadkins, Elaine M. Wainwright, Randi Jones Walker, Dewey D. Wallace, Jerry Walls, Michael J. Walsh, Philip Walters, Janet Walton, Jonathan L. Walton, Wang Xiaochao, Patricia A. Ward, David Harrington Watt, Herold D. Weiss, Laurence L. Welborn, Sharon D. Welch, Timothy Wengert, Traci C. West, Merold Westphal, David Wetherell, Barbara Wheeler, Carolinne White, Jean-Paul Wiest, Frans Wijsen, Terry L. Wilder, Felix Wilfred, Rebecca Wilkin, Daniel H. Williams, D. Newell Williams, Michael A. Williams, Vincent L. Wimbush, Gabriele Winkler, Anders Winroth, Lauri Emílio Wirth, James A. Wiseman, Ebba Witt-Brattström, Teofil Wojciechowski, John Wolffe, Kenman L. Wong, Wong Wai Ching, Linda Woodhead, Wendy M. Wright, Rose Wu, Keith E. Yandell, Gale A. Yee, Viktor Yelensky, Yeo Khiok-Khng, Gustav K. K. Yeung, Angela Yiu, Amos Yong, Yong Ting Jin, You Bin, Youhanna Nessim Youssef, Eliana Yunes, Robert Michael Zaller, Valarie H. Ziegler, Barbara Brown Zikmund, Joyce Ann Zimmerman, Aurora Zlotnik, Zhuo Xinping
- Edited by Daniel Patte, Vanderbilt University, Tennessee
-
- Book:
- The Cambridge Dictionary of Christianity
- Published online:
- 05 August 2012
- Print publication:
- 20 September 2010, pp xi-xliv
-
- Chapter
- Export citation