No effects of synchronicity in online social dilemma experiments: A registered report

Abstract

Online experiments have become a valuable research tool for researchers interested in the processes underlying cooperation. Typically, online experiments are asynchronous, participants complete an experiment individually and are matched with partners after data collection has been completed. We conducted a registered report to compare asynchronous and synchronous designs, where participants interact and receive feedback in real-time. We investigated how two features of synchronous designs, pre-decision matching and immediate feedback, influence cooperation in the prisoners dilemma. We hypothesized that 1) pre-decision matching (assigning participants to specific interaction partners before they make decisions) would lead to decreased social distance and increased cooperation; 2) immediate feedback would reduce feelings of aversive uncertainty and lead to increased cooperation; and 3) individuals with prosocial Social Value Orientations would be more sensitive to the differences between synchronous and asynchronous designs. We found no support for these hypotheses. In our study (N = 1,238), pre-decision matching and immediate feedback had no significant effects on cooperative behavior or perceptions of the interaction; and their effects on cooperation were not significantly moderated by Social Value Orientation. The present results suggest that synchronous designs have little effect on cooperation in online social dilemma experiments.

1 Introduction

Online experiments have become a valuable research tool for psychologists and economists interested in human cooperation (Arechar et al., 2018; Horton et al., 2011). Online experiments offer potential advantages, such as large sample sizes (Hauser et al., 2016) and access to diverse participants (Nishi et al., 2017); and increase the feasibility of multi- and cross-national studies (Dorrough & Glöckner, 2019; Romano et al., 2017). When social dilemmas experiments are conducted online, researchers often use asynchronous designs, where participants complete an experiment individually and are matched with partners after data collection has been completed. Asynchronous experiments differ from synchronous experiments in two important ways: First, participants are not matched to a specific partner before decisions are made. Second, participants do not receive immediate feedback on the consequences of their choices. The present research tests whether pre-decision matching and immediate feedback influence cooperation, and whether the effects of these design features are moderated by individual differences in prosocial preferences. The proposed research contributes to our understanding of how common research methods influence psychological processes and behavior in online social dilemmas experiments.

1.1 Conducting Social Dilemmas Experiments Online

Social dilemmas are situations where individuals must make a choice between pursuing self-interest and the collective good (Dawes, 1980), and the study of social dilemmas is an important point of intersection for researchers in the behavioral sciences (Van Lange et al., 2013). As the use of online samples in psychological research has grown (Birnbaum, 2000; Buhrmester et al., 2011; Gosling et al., 2004), many social dilemmas researchers have begun to rely on online participant pools such as Amazon’s Mechanical Turk (mTurk) and Prolific Academic.

Importantly, online social dilemmas experiments also produce valid data: Amir et al. (2012) found that online behavior in typical economic games (e.g., the public good game, the trust game, the ultimatum game, and the dictator game) resembles behavior observed in laboratory experiments (e.g., proposers in the ultimatum game reject unfair offers, and reciprocity in the trust game is proportional to the initial level of trust). Arechar et al. (2018) also demonstrated that online participants can be used to study behavior in repeated games: as in laboratory studies, cooperation deteriorated in later rounds of a repeated interaction, but was bolstered by introducing the possibility of peer punishment.

There are also potential limitations to online experiments. Although data quality is generally adequate (Buhrmester et al., 2011), experimental manipulations may be affected by the non-naivete and prior experience of online participants (Chandler et al., 2014). Illustrating this problem, Rand et al. (2014) found that manipulations of time pressure in social dilemmas became progressively less effective over time, in part because more participants developed prior experience with experimental games. Similarly, Chandler et al. (2015) found that effect sizes in a range of psychological tasks became less pronounced among experienced participants. However, the effects of participant sophistication can be attenuated by presenting participants with new variants of existing paradigms (Rand et al., 2014).

1.2 Synchronicity in Online Experiments

Although synchronous experiments, where participants have live interactions and receive feedback in real-time, are possible using software programs such as oTree (Chen et al., 2016) and classEx (Arechar et al., 2018), most online experiments are asynchronous. To illustrate this point, we surveyed articles published in the Judgment and Decision Making journal over an eight-year period (2013–2020): There were 40 published studies using online samples to study behavior in social dilemmas, and all of these studies were conducted using asynchronous methods. No studies that we know of have systematically compared behavior in synchronous and asynchronous online experiments. In this section, we review research suggesting that behavior may be affected by two defining features of fully synchronous experiments, pre-decision matching and immediate feedback.

1.2.1 Pre-decision Versus Post-decision Matching

In synchronous experiments, participants are assigned to specific partners before any decisions are made; in asynchronous experiments, partner assignment happens only after data collection is complete. We predicted that pre-decision matching would reduce the perceived social distance between participants. This prediction is motivated by research on charitable giving and the identifiable victim effect: people are more willing to help a single victim compared to a group of statistical victims (Kogut & Ritov, 2005a, 2005b). Critically, this effect does not depend on the specific characteristics of the individual victim (Kogut & Ritov, 2005a), and it can even occur when the recipient of help remains unidentified (Lee & Feeley, 2016; Small & Loewenstein, 2003). Arguably, people experience a stronger emotional connection to the recipient of help when the recipient is identified as any specific person (Small & Loewenstein, 2003; Small et al., 2007). In social dilemmas, we anticipated that assigning participants to interact with specific partners before they make decisions would reduce perceived social distance.

Additionally, other work suggests that pre-decision matching may reduce distance by leading people to interpret their interactions as social exchanges, rather than abstract reasoning problems: Research on the strategy method, which requires participants to make conditional decisions for all possible situations (Brandts & Charness, 2011), suggests that people sometimes become less trusting (Murphy et al., 2006) and less trustworthy (Casari & Cason, 2009) when social decisions are presented abstractly. Subtle social cues, such as referring to other players as partners versus opponents (McCabe et al., 2003) or describing an interaction as a community game versus Wall Street game (Liberman et al., 2004), can encourage prosocial behavior by making the norm of reciprocity salient. In the same way, pre-decision matching may change how participants construe their decisions, leading them to feel closer and more interconnected with their interaction partners.

We expected that pre-decision matching would reduce social distance, and that reduced social distance, in turn, would result in increased cooperation. People are generally more prosocial towards those they feel close to (Jones & Rachlin, 2006, 2009), and this increase in cooperation can even occur when proximity is based on arbitrary procedures, such as the minimal group paradigm (Balliet et al., 2014; Goette et al., 2006). In summary, our first prediction was that pre-decision matching would reduce social distance and, in turn, increase cooperation among strangers.

1.2.2 Immediate Versus Delayed Feedback

In synchronous experiments, participants immediately receive feedback on the outcomes of their decisions; in asynchronous online experiments, there is a delay, sometimes hours or days, before participants learn about interaction partners’ decisions and their final payoffs for the experiment. Delaying feedback adds a temporal dimension to the social dilemma, and previous studies have found that cooperation is more difficult to sustain when potential outcomes are projected into the future (Joireman et al., 2004; Kortenkamp & Moore, 2006).¹

We predicted that immediate feedback would increase cooperation by reducing feelings of aversive uncertainty related to the fear of exploitation. Undesirable outcomes, such as losing money or receiving an electric shock, are perceived as worse when they are projected into the future (Loewenstein, 1987). For risky decisions, delayed feedback affects the subjective perception of the likelihood and impact of negative outcomes, and thus makes people more likely to select relatively safe options (Kogler et al., 2016; Muehlbacher et al., 2012). In social dilemmas, defection may be seen as “safer” than cooperation because choosing defection eliminates the possibility of the worst possible outcome (i.e., the sucker’s payoff). Therefore, our second hypothesis was that immediate feedback would reduce feelings of aversive uncertainty and lead to increased cooperation.

1.2.3 Synchronicity and Social Value Orientation

In addition to considering the group-level effects of synchronous experiments, we also investigated whether the effects of synchronous designs were moderated by individual differences in Social Value Orientation (SVO) (Murphy & Ackermann, 2014; Van Lange, 1999). We hypothesized that prosocial individuals (i.e., those with stronger preferences to maximize joint outcomes or outcome equality) would be more likely to be affected by the distinction between synchronous and asynchronous experiments. Compared to fully self-focused individualists, prosocials are more sensitive to situational cues (Bogaert et al., 2008) and are more likely to adapt their expectations and behavior based on context (Van Lange, 1999). In other words, prosocials cooperate in social dilemmas when it can be justified by the constraints of the situation; individualists, on the other hand, tend to be unconditionally self-interested (Epstein et al., 2016; Yamagishi et al., 2014). Thus, we expected that pre-decision matching and immediate feedback would have stronger (positive) effects on cooperation for individuals with stronger prosocial preferences.

1.3 Overview of Hypotheses

We conducted an experiment comparing behavior in synchronous versus asynchronous dilemmas. We examined how two features of synchronous experiments, pre-decision matching and immediate feedback, influenced cooperation. We predicted that pre-decision matching and immediate feedback would both increase cooperation. More precisely, we expected that pre-decision matching would reduce feelings of social distance and that immediate feedback would reduce feelings of aversive uncertainty. In addition to considering the group-level effects of synchronous experiments, we also tested whether individual differences in SVO moderated their effects on behavior. We hypothesized that pre-decision matching and immediate feedback would have stronger positive effects on the behavior of individuals with prosocial preferences. Our Stage 1 report can be viewed at https://osf.io/7qnej/?view_only=d79a1ba27cd34a01b0d14fa0b8cb03a2.

2 Method

2.1 Participants

Power Analysis

We conducted power analyses using G*Power 3.1 (Faul et al., 2009) to identify the number of participants needed to detect a small effect (ϕ =.1) using a Chi Square test with 3 groups, 80% power, and α = .05: minimum N = 964. We adjusted this estimate based on an expected dropout rate of roughly 10%.² Our total planned sample size was N = 1,200.

We recruited 1238 participants from Prolific Academic. We ended up with slightly more participants than expected because Prolific sometimes classified participants as “timed out” while they were still completing the experiment. The average age was 26.64 years (SD = 9.46); and there were 648 men, 383 women, 12 non-binary participants, and 195 participants who did not report genders.

Recruitment

We conducted twelve experimental sessions with 100 available spaces per session. Sessions were launched on weekdays at 2:00PM CEST. The first three sessions were conducted in November 2020, and the remaining nine sessions were conducted in January 2021. Participants were prevented from completing the study more than once using Prolific’s “previous study” filter. Participants received a show up payment of £1.50 each, and those who finished the experiment also received bonus payments based on their choices (£0.50 to 4.00 per person).

2.2 Materials and Procedure

The experiment was administered using oTree (Chen et al., 2016). The experiment did not involve any deception.

Prisoners Dilemma

Participants made decisions in a binary choice prisoners dilemma. Each participant was assigned to a partner and chose Keep or Transfer: If both players choose Keep, then they received 100 points each; If both players chose Transfer, then they received 200 points each; If one player chose Keep and the other chose Transfer, then they received 300 points and 0 points, respectively. After reading the instructions, participants were presented with four comprehension questions (Example: “If both you and the other participant choose “Transfer”, how many points will you receive?”). The majority of participants (75%) answered all four questions correctly; and rates of accuracy did not differ across experimental conditions: asynchronous = 75%; partially synchronous = 75%; synchronous = 73% (χ ²(2) = 0.55, p = .76). Our primary analyses included all participants. Following our pre-registration, we also conducted supplemental analyses using only data from participants who answered all four questions correctly.

The full instructions and experiment materials are reported in the Appendix.

Proposed Mediators

We hypothesized that social distance and aversive uncertainty would mediate the effects of pre-decision matching and immediate feedback (respectively) on cooperation. The items measuring these constructs are reported in Table 1a. We randomized whether participants responded to the proposed mediators immediately before or immediately after they made decisions in the prisoners dilemma, and this randomization occurred at the session level.

Table 1: Summary of hypothesized mediators, post-decision measures, and post-feedback measures

Post-decision Measures

After cooperation decisions were made, participants were presented with a series of questions before they received feedback on the outcome of the interaction. We measured expectations of cooperation in their interaction partners; their confidence in these expectations; and feelings of anticipated satisfaction and regret. These items are reported in Table 1b. After completing these post-decision measures, participants then completed the 6-item slider measure of SVO (Murphy & Ackermann, 2014). In this measure, each participant was asked to make a series of hypothetical allocation decisions, where they decided how many points to share with an anonymous interaction partner.

Post-feedback Measures

After participants learned about their partners’ choices and the outcome of the interaction (including information about their bonus payment), they responded to a series of items before concluding the experiment. To measure participants’ reactions to feedback, we asked participants how satisfied they were with the outcome of the interaction and whether they regretted their choice in the interaction. Participants also played a mini-dictator game, where they decided whether to share additional money with the other participant from the prisoners dilemma. We also measured how much participants enjoyed the experiment, whether they believed that they were interacting with a real person in the interaction, the perceived fairness of the interaction, and basic demographics questions. The complete measures are included in Table 1c.

Experimental Conditions

Participants were assigned to partners based on the time at which they begin the experiment. There were three experimental conditions (summarized in Table 2): asynchronous (post-decision matching with delayed feedback); partially synchronous (pre-decision matching with delayed feedback); and fully synchronous (pre-decision matching with immediate feedback). Experimental conditions were assigned at the level of session, with four sessions for each condition.

Table 2: Summary of experimental conditions and descriptive statistics

Note: “Matching rate” refers to the proportion of participants who were successfully assigned to partners during the first part of the experiment. “Follow up rate” refers to the proportions of participants that completed the second part of the study (which was administered one week after the first part).

Pre- vs post-decision matching

Participants in the pre-decision matching conditions were assigned to partners before they made decisions in the prisoners dilemma. If no partner was immediately available, then participants were redirected to a waiting screen where they were asked to wait for a period of up to five minutes. If no partner could be located during that time frame, then participants had the option to continue waiting for another five minutes or to terminate the experiment, in which case they received the show up payment (but they did not receive any bonus payment). Across sessions, almost all participants (0.98) were successfully matched with partners. The average waiting time was 9.66 seconds (SD = 66.02).

In the post-decision matching condition, participants were assigned to partners after they had already made decisions in the prisoners dilemma.

Immediate vs delayed feedback

In the immediate feedback condition, participants learned about the outcome of the prisoners dilemma as soon as both players made their decisions and answered the post-decision questions. In the delayed feedback conditions, participants were contacted via Prolific messages one week after the initial experiment session and invited to complete the study. A majority of participants (80%) from the delayed feedback conditions completed the second part of the study.

2.3 Deviations from pre-registered protocol

There were five ways in which our study deviated from our pre-registered protocol:

1. 1. We recruited participants from Prolific Academic rather than MTurk. We made this switch given growing concerns about the presence of bots and deteriorating data quality on MTurk (Chmielewski & Kucker, 2020).

2. 2. We reduced participant payments from $3 to £1.50. Given the duration of our study (typical completion time of 5–8 minutes), $3 would have been an unusually high rate-of-payment on the Prolific platform (typical studies pay £0.10 to 0.20 per minute).

3. 3. Our original plan was to administer multiple sessions per day. We decided to run only one session per day during the peak usage hour on Prolific (https://researcher-help.prolific.co/hc/en-gb/articles/360011657739-When-are-Prolific-participants-most-active-). The purpose of this change was to increase the number of available participants, and to maximize the chance that participants would be successfully matched in real-time.

4. 4. We randomized whether mediation questions were presented before (or after) the prisoners dilemma. Originally, we planned to randomize this factor at the level of dyad. Instead, we decided to implement this randomization at the session level.

5. 5. After we collected data for sessions 1–3 of the study, a participant informed us about a potential problem. Some participants were able to view their condition assignments in the Internet browser’s URL bar. We paused further data collection until we were able to fix this issue. Note that excluding participants from the first three sessions did not change any of our results.

3 Results

3.1 Primary Analyses

Our primary analyses focused on the effects of pre-decision matching and immediate feedback on cooperation, social distance, and feelings of aversive uncertainty. Descriptive statistics for these variables by condition are shown in Figure 1. Following our analysis plan, we conducted two-tailed tests with α = .05.

Figure 1: Levels of cooperation (A), social distance (B), and aversive uncertainty (C) by experimental condition. Plots B and C use violin plots and box plots to show distributions of responses within each condition. Bold lines indicate median responses, and box widths indicate values that lie within the first and third quartiles.

Pre-decision matching

Our first prediction was that pre-decision matching would increase cooperation via reduced social distance. First, we used a logistic regression to compare the rates of cooperation in the asynchronous and partially synchronous conditions (-.5 = asynchronous; +.5 = partially synchronous). Pre-decision matching had no significant effect on cooperation (b = −0.19, SE = 0.15, p = .20). Then, we compared the levels of social distance between the two conditions. Pre-decision matching did not significantly affect social distance (b = 0.28, SE = 0.16, p =.084). Social distance was, however, negatively associated with cooperation (b = −0.08, SE = 0.03, p = .012).

Immediate feedback

Our second hypothesis was that immediate feedback would reduce feelings of uncertainty and lead to increased cooperation. We compared the rates of cooperation in partially synchronous and fully synchronous conditions (-.5 = partially synchronous; +.5 = fully synchronous). Immediate feedback had no significant effect on cooperation (b = −0.05, SE = 0.15, p = .75). Then, we compared the levels of aversive uncertainty across conditions. Immediate feedback had no significant effect on aversive uncertainty (b = −0.04, SE = 0.20, p = .83). Aversive uncertainty was not significantly associated with cooperation (b = 0.05, SE = 0.03, p = .052).

Social Value Orientation

Our third hypothesis was that the effects of synchronous experiments would be moderated by individual differences in SVO. More specifically, we expected that pre-decision matching and immediate feedback would have stronger effects on cooperation for individuals with stronger prosocial orientations. To test this prediction, we estimated a series of regression models predicting cooperation, social distance, and aversive uncertainty. Each model included the following variables as predictors: pre-decision (vs. post-decision) matching; immediate (vs. delayed) feedback; Social Value Orientation angle (mean centered); and two SVO by experimental condition interaction terms. The results are reported in Table 3. Reassuringly, SVO was positively correlated with cooperation, but we found no support for the predicted SVO-by-condition interactions.

Table 3: The interactive effects of Social Value Orientation, pre-decision matching, and immediate feedback

3.2 Secondary Analyses

Robustness checks

To test the robustness of our primary analyses, we conducted two sets of alternative analyses: First, we estimated models including participants demographics (i.e., age, gender, English proficiency, and income) as covariates. Then, we estimated the models outlined in the previous section using only participants who correctly answered all four comprehension questions. Results were consistent with our primary analyses (See Appendix).

Post-decision measures

We also conducted a series of exploratory analyses of the effects of pre-decision matching and immediate feedback on four post-decision measures: expectations of cooperation, confidence in expectations, anticipated satisfaction, and anticipated regret. The results are reported in Table 4. There were no significant effects of either pre-decision matching or immediate feedback.

Table 4: The effects of pre-decision matching and immediate feedback on post-decision measures

Post-feedback measures

Next, we examined the effects of pre-decision matching and immediate feedback on six outcomes measured after participants learned about the outcome of the prisoners dilemma: behavior in a mini dictator game; experienced satisfaction with the outcome of the prisoners dilemma; experienced regret about choices in the prisoners dilemma; enjoyment of the experiment; the perceived fairness of the experiment; and the perceived realism of the experiment. We also included participant payoff (the amount of points earned in the prisoners dilemma, scaled from 0 points = −1 to 400 points = +1) as a predictor, as well as two payoff by experimental condition interaction terms. The results are reported in Table 5. Pre-decision matching and immediate feedback had little, if any effect, on post feedback outcomes. Out of 24 regression coefficients involving experimental condition, one was significant at p < .05. Not surprisingly, participant payoff had large effects on post-feedback outcomes: Participants who earned more in the prisoners dilemma were more altruistic in the dictator game; were more satisfied with their outcomes and had less regret about their choices; and enjoyed the experiment more and perceived it as more fair.

Table 5: The effects of payoff and experimental condition on post-feedback measures

* p<.001;

** p<.01;

*** p<.05.

Waiting time

To conclude, we investigated whether waiting time (in the two pre-decision matching conditions, N = 787) was correlated with cooperation, perceived closeness, aversive uncertainty, or perceived realism. To account for the non-normality of the waiting time data, we log-transformed it. There were no significant correlations (r’s < .05, p’s > .18).

4 Discussion

Researchers often use asynchronous online experiments to examine the psychological processes underlying cooperation. However, it is unclear how asynchronous interactions differ psychologically from synchronous interactions, where participants interact in real-time and receive immediate feedback. We examined the effects of two salient features of synchronous experiments, pre-decision matching and immediate feedback, on cooperation in the prisoners dilemma. We found that pre-decision matching and immediate feedback had no significant effects on cooperation (or on how participants perceived the interaction). The present results suggest that synchronous design features have little effect on behavior in online experiments measuring cooperation.

4.1 Synchronous Online Experiments

In our study, we tested three hypotheses about the effects of synchronicity on cooperation: Our first hypothesis was that pre-decision matching would increase cooperation via reduced social distance. Pre-decision matching had no effects on cooperation or perceived social distance; however, social distance was associated with decreased cooperation. Previous studies have found consistent evidence that people help and cooperate with socially proximate interaction partners (Jones & Rachlin, 2006, 2009). Here, we found that merely assigning participants to a specific interaction partner is not sufficient to create feelings of social proximity.

Our second hypothesis was that immediate feedback would increase cooperation by reducing participants’ feelings of aversive uncertainty about the possibility of exploitation. Immediate feedback had no effects on cooperation and aversive uncertainty, and aversive uncertainty was not significantly associated with cooperation. Introducing a (one-week) delay in outcome does not substantially affect behavior in the prisoners dilemma.

Our third hypothesis was that SVO would moderate the effects of pre-decision matching and immediate feedback on behavior. We found that SVO was correlated with cooperation and negatively correlated with social distance. However, SVO did not moderate the effects of synchronicity. This is unsurprising, as participants were generally insensitive to the differences between experimental conditions.

We also examined the effects of synchronicity on perceptions of the realism of the experiment: Interestingly, having participants engage in real-time interactions had no significant effects on the perceived realism of the experiment. Responses to our “perceived realism” question were relatively close to the midpoint, 5 out of 10, across all conditions; and participants did not feel socially close to their interaction partners. In terms of realism, synchronous experiments do not convey much advantage over asynchronous experiments. This may point to a general limitation of experiments using economic games, rather than a specific limitation of asynchronous experiments. Other design features likely have larger effects on the extent to which participants perceive a social dilemma experiment as a real interaction. To increase realism, researchers may need to provide participants with identifying information about their interaction partners (Evans & Krueger, 2016), or allow participants to communicate directly during the experiment (Dawes et al., 1977).

4.2 Limitations

In our study, we measured behavior using one social dilemma, the dyadic prisoners dilemma. We cannot rule out the possibility that synchronous designs would matter more in other settings. For example, synchronicity could matter more in sequential (rather than simultaneous) interactions, such as the trust game or the ultimatum game. Previous research has argued that contextual cues play an important role in activating the norm of reciprocity in sequential exchanges (McCabe et al., 2003). On the other hand, research suggests that people show high levels of consistency in their behavior across different types of experimental games (Yamagishi et al., 2013). Researchers interested in pursuing these questions further should be prepared for the possibility that partner matching and feedback effects are small and relatively difficult to detect.

Additionally, our manipulation of delayed feedback focused only on one time interval, immediate feedback versus a one-week delay. We focused on the interval of one-week because this is a typical delay of payment in online experiments. However, we cannot rule out the possibility that longer time delays (one-month or longer) could have effects on aversive uncertainty and cooperation. It is also important to consider whether our choice of payoff stakes affected our results: We used standard payoff stakes for online experiments (£0.50 to 4.00), which were equivalent to the payments participants would receive 5 to 40 minutes of work on Prolific. It is possible, but not likely, that larger payoff stakes would increase participants’ sensitivity to pre-decision matching and delayed feedback (Amir et al., 2012).

Finally, it is important to note that our results may have been influenced by participant non-naivete (Chandler et al., 2014). Previous studies have raised the possibility that experimental manipulations have weaker effects on participants once they have become familiar with a paradigm (Chandler et al., 2014; Rand et al., 2014). This is a general problem for online experiments conducted on platforms like Prolific or mTurk. Pre-decision matching and immediate feedback may have larger effects on participants who are generally unfamiliar with social dilemmas. Moreover, some participants may have been skeptical about the veracity of our study. Concerns about deception and the contamination of shared subject pools are unavoidable (Hertwig & Ortmann, 2008). Indeed, a small number of participants sent messages indicating that they did not believe they were actually partnered with other participants. However, we found no evidence that belief in the realism of the study was correlated with behavior in the prisoners dilemma.

4.3 Advice for Social Dilemmas Researchers

At this point, researchers interested in social dilemmas may wonder whether it is worthwhile to conduct synchronous experiments: On the positive side, our study demonstrates that it is feasible to conduct large scale studies involving real-time partner matching and multiple time measurements. Nearly all participants were successfully matched to partners (98%) and participant retention was relatively high across the two waves of the study (80%). At the same time, synchronous experiments also require a substantial time investment (compared to asynchronous experiments conducted using Qualtrics, or similar software).

While we did not find any effects of synchronicity on behavior, allowing for the possibility of synchronous interactions opens up new potential research questions. The study of social dilemmas has largely focused on anonymous, one-shot interactions without communication and minimal social information (Thielmann et al., 2020; Van Lange et al., 2013). However, interactions with strangers account for a relatively small percentage (~10%) of daily social interactions (Columbus et al., 2021). Arguably, this focus has been influenced by the relative ease of running asynchronous experiments. As our study demonstrates, software packages such as oTree (Chen et al., 2016) are making it easier for researchers to conduct high-powered studies of cooperation that go beyond zero-acquaintance interactions.

4.4 Conclusion

How does synchronicity influence perception and behavior in online experiments measuring cooperation? We found that pre-decision matching and immediate feedback had no significant effects on behavior in the prisoners dilemma or on how participants perceived the interaction. The present results suggest that synchronous designs and asynchronous designs can produce similar results in studies of online cooperation.