Returning aggregate results of clinical trials: Empirical data of patient preferences

Abstract

Introduction:

The purpose of this research was to understand the preferences of patients receiving integrative medicine services for return of aggregate study results.

Methods:

A brief online survey (survey 1) was sent to 341 cancer patients receiving integrative medicine interventions; subsequently, a minimally revised survey (survey 2) was sent to 812 individuals with various medical conditions who had been either research participants in integrative medicine studies (n = 446) or patients (n = 346) of mind–body medicine.

Results:

Feedback to a model plain language summary was elicited from survey 1 and survey 2 respondents. Seventy-seven survey recipients (23%) responded to survey 1, and 134 survey recipients (17%) responded to survey 2. The majority of respondents to the surveys were female and 51–70 years of age. Ninety percent of responders to survey 1 and 89% of responders to survey 2 indicated that researchers should share overall results of a study with participants. In terms of the means of result distribution, 37%–47% preferred email, while 22%–27% indicated that, as long as the results are shared, it did not matter how this occurred. Of 38 survey 1 respondents who had previously participated in a clinical trial, 37% had received the results of their study. In survey 2, 63 individuals indicated that they previously participated in clinical trials, but only 16% recalled receiving results.

Conclusions:

These results confirm that the majority (89%–90%) of integrative medicine patients are interested in receiving the results of clinical trials. The majority (82%–94%) of respondents felt the model plain language summary of results provided was helpful.

Introduction

There is a growing appreciation that transparency in clinical trials and engagement with clinical trial participants will advance science and increase public trust [1, 2]. Recognizing the rights and interests of participants and the public to information about clinical trials, the European Union requires that all sponsors of clinical trials submit a summary of results that is understandable to a layperson [3, 4]. The United Kingdom has put forward similar guidance [5]; and in the United States, the National Academy of Medicine [6], the Patient Centered Outcomes Research Institute [7], and the National Institutes of Health [8, 9] have recognized the importance of this need and, as a result, the mandate for greater transparency has gained momentum in recent years. In the United States, however, federal guidance and policy have stopped short of requiring the return of plain-language summaries to participants. Research results of interventional and selected other studies are required to be posted publicly to the federal database ClinicalTrials.gov [10]. However, results are typically not written in plain language, and there is no mandate that results be returned directly to study participants.

Individuals volunteer to take part in a trial, at some personal risk and with no guarantee of benefit, not only for access to innovative treatments but also for altruistic reasons to help others. A majority of participants anticipate receiving the results of the trials in which they participated. A 2008 literature review across 15 separate studies found that a median of 90% of clinical trial participants wished to be told of the results of the clinical trial in which they participated [11]. There is some experience with return of results. A 2011 study [12] provided plain-language summary results in three communication formats (printed report, webpage, and telephone hotline): 90% of those surveyed reviewed the printed report, while only 5% reviewed the webpage. The same study also found that 91% of respondents, after reading the plain-language summary, felt they understood the results of their clinical trial “very well” (57%) or “somewhat well” (34%). However, few study participants understand that it may be years before such results are available, and fewer still receive results at all [13]. Importantly, one survey of patients undergoing a general medical evaluation demonstrated that 68% would not participate in future research if not informed about the results of a study in which they participated [14].

Not much is known about preferences for returning summary results to patients who participate in studies of integrative medicine. Since the therapeutic approaches and the provider–patient interaction differ in integrative medicine compared with conventional biomedicine, we questioned whether there would be a difference in preferences for returning results to integrative medicine patients. We therefore conducted an exploratory study consisting of two surveys of patients and research participants of integrative medicine. Survey 1 focused only upon cancer patients, while survey 2 included patients with a variety of conditions. Both surveys addressed expectations and preferences for sharing of aggregate clinical trial results.

Materials and Methods

The aim of the surveys was to understand the preferences of, and to test a template for, sharing study results with participants of integrative medicine therapies at a regional cancer center and of a mind–body service within a general hospital.

As a model for a plain-language summary of aggregate research results, we used a published template that had been developed and vetted by a multi-stakeholder group, including patient advocates. That template had been published [15] and its features incorporated into the European Union directive on plain-language summaries [16]. Two populations receiving integrative medicine services were surveyed. For survey 1, the results of a study [17] investigating the role of acupuncture in head and neck cancer treated by radiation therapy and chemotherapy were adapted to a plain-language summary (see Supplementary Material, Appendix 1). For survey 2, using the same template as for survey 1, the results of a pilot study [18] investigating the efficacy of a mind–body intervention on inflammatory disease markers and self-reported quality of life in patients with irritable bowel syndrome and inflammatory bowel disease were adapted to a plain-language summary (see Supplementary Material, Appendix 2). Neither of these studies had programmatically shared aggregate results with study participants.

Each plain-language summary was prepared by a member of the study team (C.E.A.) and was reviewed and edited by another study team member (B.E.B.). Both of these individuals had been members or leaders of the multi-stakeholder group that developed the plain-language summary template. Each summary was then reviewed for accuracy and approved by the research teams of integrative medicine studies. Each plain-language summary began with a note of appreciation and a disclaimer that newer information may be available since the summary was completed. Health literacy and numeracy principles [19] were used throughout. For instance, for survey 1, the title “Acupuncture for dysphagia after chemoradiation therapy in head and neck cancer: a randomized sham-controlled study,” [20, 21] was simplified to “A pilot study of acupuncture treatment for swallowing problems in head and neck cancer patients.” The simplified titles were followed by a one-sentence description of the study and an explanation of why the study was done. Study information in the template contained a summary of the study group and treatments, timeframe, and location of the sites. Next, the template described the study design and included a section on “side effects.” The study results were then summarized with the disclaimer that they were limited to the particular study of the people who enrolled. Final comments included the official name of the study, the ClinicalTrials.gov unique identifier, the sponsor of the study, where further information could be found, and who to contact for additional information.

A short online quantitative and qualitative survey (see Supplementary Material, Appendix 3) was prepared, consisting of 10 questions to ask participants about their general knowledge of integrative medicine and clinical research, whether they had ever participated in a clinical trial, and, if so, whether they heard about the results of the study. We asked whether researchers, generally, should share the overall results of a study and the optimal format for doing so. We then requested respondents to review the prepared summary and asked whether it was helpful or unhelpful and to explain their answer. Survey 2 also included questions about the participants’ medical condition and the type of integrative medicine therapies they had received.

The Dana-Farber Cancer Institute’s Institutional Review Board (survey 1) and Partners HealthCare Institutional Review Board (survey 2) each determined that the study met the criteria for exemption. The cover email sent to prospective participants explained that we interpreted completion of the survey as consent to participate. No individual participant-level identifiers were retained for either survey.

For survey 1, invitations were sent in June 2016 by email to 341 New England residents who had been seen at the Leonard P. Zakim Center for Integrative Therapies and Healthy Living at least twice in the last two years. Forty-five of the 341 invitations included active program participants who had attended group programs or integrative clinic services for a year or more on a consistent basis. Two email reminders were sent, and the survey was closed thereafter. For survey 2, invitations were sent by email in four separate batches between April and May 2017 to a total of 812 individuals who had been either research participants (n = 446) or patients (n = 346) at the Benson-Henry Institute for Mind Body Medicine at Massachusetts General Hospital. One reminder was sent to each recipient, and the survey was closed to data collection in June 2017.

The surveys were administered via REDCap (https://www.project-redcap.org/), a secure web application for building and managing online surveys and databases that provides descriptive statistics for quantitative questions. Prior to analyzing the qualitative responses, the study team received biostatistical advice from the Harvard Clinical and Translational Science National Center on resources and approaches for coding the qualitative responses and eventually chose to use a spreadsheet to record the categories and codes for each response. Subsequently, responses to five qualitative questions in survey 1 and four qualitative questions in survey 2 were coded independently by two coders based on an open coding approach [22]. The first coder (C.E.A.), a PhD in educational studies with qualitative research experience, drafted codes based on the responses. The second coders had a master-level (survey 1) and a bachelor-level (survey 2) degree, respectively, and were introduced to coding categories by the first coder, but the coding of any individual response was not discussed. The inter-rater reliability was 94.5% (survey 1) and 99.7% (survey 2), respectively. An independent third person (B.E.B.) with decades of experience in conducting and overseeing research resolved any discrepancies.

Results

Demographics of Respondents

The demographics of the responders to the two surveys are given in Table 1. The response rates were 23% (n = 77) for survey 1 and 17% (n = 134) for survey 2. In both surveys, the majority of respondents were female and 51–70 years old (see Table 1). All participants in survey 1 were cancer patients treated at the Dana-Farber Cancer Institute. In survey 2, of those who responded to the question of diagnosis (n = 109), 42% (n = 46) had diseases of the nervous system (e.g., anxiety, attention-deficit disorder). Seventeen respondents (survey 2) identified themselves as current cancer patients.

Table 1. Demographics

^a Missing data from two participants in study 1, 9 participants in study 2.

^b Missing data from one participant in study 1, 10 participants in study 2.

^c Multiple answers were possible in study 2.

Respondents’ Self-Assessment of General Knowledge and Clinical Research Experience

Ninety-two percent (survey 1) and 73% (survey 2) of respondents indicated they were “very informed” or “somewhat informed” about alternative, complementary, and/or integrative medicine (see Table 2). Seventy-eight percent and 72% of respondents, respectively, indicated they were “very informed” or “somewhat informed” about clinical research (Table 2).

Table 2. Respondents’ self-assessment, clinical research experience, and preference for sharing results

^a Missing data from one participant in study 1, 10 participants in study 2.

^b Missing data from one participant in study 1, nine participants in study 2.

^c Missing data from four participants in study 1, 10 participants in study 2.

^d Missing data from one participant in study 1, nine participants in study 2.

Fifty-two percent (n = 38) of respondents in survey 1 and 51% (n = 63) of respondents in survey 2 indicated they participated in a clinical trial (Table 2). The majority of those who participated in a clinical trial reported they did not learn about study results. Of those who did learn of results, email messages and in-person meetings with study personnel were the most frequent means of receiving study results (see Table 3).

Table 3. Experiences and preferences of sharing results by prior clinical trial participants

* Multiple answers were possible; 14 participants responded in study 1, 10 participants responded in study 2.

Preference for Sharing Results

Whether or not individuals had participated in clinical research previously, 90% (n = 68) of respondents in survey 1 and 89% (n = 111) of respondents in survey 2 indicated that researchers should share the overall results of a study with participants (see Table 2). Eleven percent (n = 8) and 10% (n = 12) of participants, respectively, were not sure. No one in survey 1 indicated that results should not be shared. Two individuals in survey 2 indicated that results should not be shared. However, one of them explained that “individuals should get their personal results back,” raising questions about the discordance between the survey question that addressed aggregate or summary study results and the intent of the answer.

Those who were not sure whether researchers should share results with participants had varying reasons:

“If it is relevant to my diagnosis, then I would like it to be shared. Otherwise I am okay not knowing, just hoping that it has helped someone.” (Survey 1)

“I think researchers shouldn’t need to spend their valuable time corresponding with study participants. That seems like a total ‘time suck’ and I’d much rather have researchers focusing on what they do best – research. On the other hand, I would be personally interested in the results.” (Survey 1)

“It depends on the kind of study. If, for example, the researchers suspect that I’m unaware of some illness or potential illness that they discovered in me, I’d appreciate their telling me to check in with my doctors and share the discovery they found in me. If the study is using my general data that I had volunteered to allow hospital researchers to use, then it’s possible that the statistical findings may be beyond my understanding.” (Survey 2)

Explanations for Sharing Results with Participants

We asked respondents to explain why they thought researchers should share results. The most frequent response in both surveys was the participants’ interest in research results – they wanted to learn what the study found. This was followed, in both surveys, by their belief that it was an ethical expectation to share results with those who participated. Collectively, participants were interested if the results had relevance to their own health. Sample responses included:

“I believe this needs to be included in the ethical requirements of informed consent, participants should be informed about being participants and researchers should then be required to inform all participants of ANY results of the study.” (Survey 1)

“The more we know, the better decisions we can make for ourselves.” (Survey 1)

“Knowing the results of a study is also beneficial to participants since it helps them be better informed about the effectiveness of a treatment. As a participant you are lending your body or experience so sharing results in my opinion is a respectful practice.” (Survey 2)

Format for Sharing Study Results

Those who indicated that researchers should share results were asked the best format for sharing study results with participants. Thirty-seven percent (n = 25; survey 1) and 47% (n = 52; survey 2) preferred email, while 22% (n = 15) and 27% (n = 30), respectively, indicated that, as long as the results are shared, it does not matter how they are shared. The complete distribution is shown in Fig. 1.

Fig. 1. Preferred format for sharing results.

The most frequent reason given in both surveys by those who chose email was its ease, low cost, flexibility, and accessibility. For instance:

“Email gets to the participants quickly, easily, individually. People are less likely to go to a website; researchers are less likely to use the time-consuming methods of on-person meetings, phone calls, etc.” (Survey 1)

Of those who preferred sharing via website, ease, accessibility, and cost were cited, similar to email. For instance:

“If the results are on a website anyone interested or belonging to a group could log in when they need to, information would not get lost or overlooked.” (Survey 2)

Of those who preferred a letter mailed to home address, the main reason was the fact that a letter can be reread and shared. For instance:

“A printed summary gives participants time to reflect on the results and something concrete to take to their doctors to further their understanding of both their present and future care.” (Survey 1)

Of those who preferred an in-person meeting, one main reason was the opportunity to dialogue and have questions answered. Many participants did not express a preference as long as the results are shared. Some respondents in survey 1 thought that the preference of study participants should be respected; some suggested utilizing two or more options for returning results; and some respondents in both surveys reiterated the importance of sharing but not the modality. For instance:

“Any format would be fine, as long as it is confidential or un-identifiable.” (Survey 2)

Feedback on Template

We asked respondents to review a three-page, plain-language summary of results of the acupuncture study (survey 1) and a three half-page summary of the mind–body intervention study (survey 2). Respondents were asked a “Yes/No” question as to whether the summary was helpful, and then were asked an open-ended question to explain their answer. Eighty-two percent (n = 62) and 94% (n = 131) of respondents, respectively, felt that the summary they read was “helpful” overall. In both surveys, respondents mentioned that it (1) provided information and knowledge about the study and results and (2) was clear, concise, and easy to understand. Respondents also mentioned that the summary gave an explanation of results (survey 1) and that the summary was encouraging to patients (survey 2).

When the question was posed in a different way, 30% (n = 23) and 16% (n = 21) of respondents, respectively, thought that something in the summary was “unhelpful” or “unclear.” In survey 1, respondents explained that the summary did not show the expected effect of treatment, provided conflicting information, or was not relevant to their personal experience.

A few respondents in both surveys suggested that the summary could be more robust and/or explain results better, while others in the same category suggested to make it more succinct. Specific suggestions included adding “what does this mean for you” and using graphs. Some suggested providing a means to answer the questions of study participants or to make it relevant to their personal experience.

Discussion

The results from these surveys demonstrate that 89%–90% of integrative medicine patients wanted researchers to share the overall results of the study with participants. The desire to receive the summary results was consistent between the two survey populations and consistent with prior results of patients in biomedical clinical trials [23]. These results confirm that patients and participants who engage in integrative medicine are as interested as participants in biomedical clinical trials in receiving the results of clinical trials.

In survey 1, 37% of integrative medicine participants had received results from the clinical trials in which they participated. In survey 2, only 16% of integrative medicine patients had received results from the clinical trials in which they participated. While these percentages differ from each other, both are higher than that reported previously in biomedical studies: a 2012 survey of 213 trial participants found that although 97% wanted to know the results of the clinical trials in which they participated, only 9% had received the results [24]. Whether the greater access to summary information reflects differences in treatment modality (integrative medicine versus biomedical interventions), a systematic bias secondary to differences in practice (later results, described here, reflecting a general, more recent trend to return results), sampling error aggravated by small numbers, or another reason is not clear. Importantly, however, a profound difference remains between the numbers of participants who wish to receive results (89%–90%) and those who actually do receive information (16%–37%) about the studies in which they participated.

The results of both surveys indicate that the preferred format for receiving study results was email (37%–47%); these findings, however, may have been biased by the fact that these surveys were conducted through email. In contrast, a previous study conducted 6–7 years earlier showed that 90% of respondents preferred the printed version of the plain-language summary [25]. In that earlier survey, email was not offered as an option for receiving results, and access to email may have changed in the time period between these studies. Importantly, in the current study, approximately one in five (22.1%) participants in survey 1 and more than one in four (27%) participants in survey 2 indicated that the format did not matter as long as the results are shared.

That participants were interested in the research results corresponds with a 2015 study that found that 73% of patients who participated in a clinical study were interested in receiving a summary of the study results [26]. In the present surveys, participants specifically stated their belief that it was an ethical expectation to share results with those who participated and cited the importance of the relevance of the results to their own health.

Some of the qualitative responses in the current surveys imply a concern about privacy in sharing results. One respondent stated plainly, “Patient privacy is very important to me” (survey 2). While the materials we prepared for the participants did not address how participant privacy would be protected, the Guidance Document on return of aggregate results prepared by the Multi-Regional Clinical Trials Center at Brigham and Women’s Hospital and Harvard (MRCT Center) does address privacy, providing, for instance, that written communications not include a return address with the name of the study on an envelope [27]. While privacy is important, returning summary results requires the maintenance of personal information beyond the time necessary to complete the study; any prolongation of holding identifiable information can increase the possibility of breach or re-identification, however minimally. Future research will need to query whether the small risk of re-identification outweighs the desire to receive summary results. It is possible that the answer will depend upon the specific study, factoring in the sensitivity of the study (e.g., HIV versus eczema treatment) and the risk of discovery (e.g., letter mailed to home versus one-on-one conversation in the clinic).

The vast majority of respondents (82%–94%) in the current study felt that the templated summary of the research study was helpful. Nevertheless, when the question was posed differently, 16%–30% of respondents indicated that something was unhelpful or unclear about the summary. Free-text responses suggested that the summary could be more robust, better explained, or more detailed, and, alternatively, that the summary could be more succinct and concise. These conflicting responses reflect a range of participant preferences and suggest that future plain-language summaries may need to be presented in a more flexible format, one in which participants who want further information can be guided (e.g., click on a link) to additional information.

In these two surveys, the majority of integrative medicine patients wished to receive summary results from the studies in which they participated; approximately 40% of the respondents preferred receiving results by email and an additional 20% did not care about the format so long as the results are shared. The majority of respondents found a model template for the return of summary results helpful.

It is important to appreciate the significant limitations of these surveys: both were conducted through clinics in the northeast United States; the surveys were conducted by email; the sample size and response rates were small; and only one model result for each study was available for review. Seventeen respondents in survey 2 identified themselves as having cancer; although unlikely, some respondents may have been counted twice. Some qualitative responses indicated that participants anticipated seeing their own individual study results rather than aggregate results. This confusion among some participants about what kind of results this survey inquired about could have led to some inaccurate responses.

Nevertheless, this exploratory study supports the contention that participants involved in integrative medicine wish to learn of results of studies in which they participate. It will be important to confirm these results in a larger study with a more diverse population, a higher response rate, and one in which internet connectivity is not required for response and in which geographic diversity (e.g., outside the US urban setting of academic institutions) is considered.

In order to prepare plain-language summaries for future studies and for use in research, resources are available to assist in the process of preparing and disseminating plain-language summaries of research results. The European Commission has provided recommendations that include a template with sample language [28]. The MRCT Center has prepared both a Guidance Document [29], which describes planning for and providing plain-language summaries throughout the process of a clinical trial, and an accompanying Toolkit [30], which includes a template for the communication of study results with study participants, sample plain language for endpoints, and a guide to non-promotional language. Summaries of sample clinical trial results are available at the Center for Information and Study on Clinical Research Participation (CISCRP) [31], and a searchable portal for trial results summaries is available at TrialScope [32]. TransCelerate Biopharma has published recommendations for drafting non-promotional plain-language summaries of clinical trial results [33]. Collectively, these resources help to respond to participant preferences. We hope that returning summary research results in a language and format understandable to the participant will further participant engagement, respond to their expressed wishes, and follow the principles of respect and autonomy.