Development and evaluation of the feasibility, validity, and reliability of a screening tool for determining distress and supportive care needs of adolescents and young adults with cancer in Japan

Abstract

Objectives

This study aims to (i) develop a screening tool for determining distress and supportive care needs of adolescent and young adult cancer patients (AYAs) based on the NCCN's Distress Thermometer and Problem List (DTPL), (ii) evaluate its feasibility, discriminant validity, and test–retest reliability in clinical settings, and (iii) report prevalence of distress and unmet needs.

Method

In the development phase, after translation of the Japanese version of the DTPL (DTPL-J) from English into Japanese and back translation, cognitive debriefing was performed. Items in the problem list were modified to better reflect AYAs’ concerns after interviews. The modified items were reviewed and accepted unanimously by healthcare professionals. In the feasibility phase, the DTPL-J for AYAs was used in a clinical setting for 3 months. Descriptive statistics of participants’ demographics, selected items, and DT scores were calculated to report prevalence of distress and unmet needs. Response and referral rates to experts were assessed to evaluate feasibility. Some items were compared with patient demographics to assess discriminant validity. Among the patients who responded at least twice, correlations between two consecutive screenings were assessed to evaluate test–retest reliability.

Results

The DTPL-J consisted of 49 items in five categories. Of 251 patients, 232 (92.4%) were provided the DTPL-J and 230 (91.6%) responded. Based on the DT cutoff of ≥4, 69 of 230 patients (30%) had high distress. Anxiety (n = 85, 36.6%) was the most commonly selected item. Primary nurses referred 45 (21.7%) patients to an attending physician or another expert. Referral rates after DTPL-J use were higher than rates before use, but the difference was not statistically significant (p = 0.06). The items compared were consistent with their social background. A positive correlation was observed between two responses for some items.

Significance of results

The feasibility, discriminant validity, and test–retest reliability of the tool were suggested.

Introduction

In the adolescent and young adult (AYA) population aged 15–39 years in Japan, approximately 20,000 people are newly diagnosed with cancer each year, or approximately 2.3% of all people diagnosed with cancer (Katanoda et al., 2017). Since the number of AYA cancer patients (AYAs) at each hospital is small and the primary cancer site varies (Ohara et al., 2018), it is difficult for medical staff to gain experience related to providing medical care and support to AYAs.

The AYA age group has worse survival than older or younger cohorts (Bleyer, 2007). The 5-year survival rate for all AYAs has improved in recent years (van der Meer et al., 2020). Some cancers have 5-year survival rates of greater than 90%, while others have not experienced any change in treatment outcomes (Park et al., 2021). In this population, there are many unique aspects of care to consider that might influence outcomes (Bleyer, 2007). These include the developmental status of the age group (Kim et al., 2018), psychosocial difficulties (Warner et al., 2016; Fardell et al., 2018), barriers to access to specialized centers (Wolfson et al., 2017), a lack of specialist care guidelines (Nakata-Yamada et al., 2016), and clinical trials (Bleyer et al., 2005) relevant to AYAs, and differences in cancer biology and chemotherapy pharmacokinetics in cancer types (Harrison, 2009; Chiaretti et al., 2013).

AYAs require more support than older adults or younger children with cancer (Close et al., 2019). They have age-unique needs related to friendship (Fladeboe et al., 2021), employment (Guy et al., 2014), education (Parsons et al., 2012), health behaviors (Deleemans et al., 2021), sexuality (Abelman and Cron, 2020; Burns et al., 2021), and social and family issues (Kirchhoff et al., 2017). However, many of these needs are unmet (Zebrack et al., 2010; Warner et al., 2016). More than 70% of AYAs reported unmet supportive care needs in Japan (Okamura et al., 2021).

To address the distress and supportive care needs of cancer patients, the Distress Thermometer and Problem List (DTPL) was developed by the National Comprehensive Cancer Network (NCCN) (Riba et al., 2019). The NCCN Guidelines for Distress Management recommend prompt evaluation of each new patient's distress and supportive care needs with the DTPL as an initial global screening. Early evaluation and screening for distress and supportive care needs lead to early and timely management of distress, which in turn improves medical management (Riba et al., 2019).

In Singapore, the DTPL was useful for identifying clinically significant psychological distress in AYAs during the early phases of their cancer journey (Chan et al., 2018). When Chan et al. (2018) administered the distress thermometer (DT) to Asian AYAs, they found that a cutoff of 4 was significantly associated with worry, depressed mood, and nervousness. When they tested the original adult problem list with AYAs, they found that relationships existed between endorsement of the items in the problem list and distress. In Australia, an AYA-specific screening tool was developed based on the DTPL. It helped clinicians support psychosocial coping of AYAs during active treatment and promoted healthy post-treatment survivorship (Palmer et al., 2014). The tool was validated in a multinational study in primarily English-speaking countries (Patterson et al., 2021b). Psychosocial support using the tool is being implemented as a national project in Australia (Patterson et al., 2021a). Although AYA-specific screening tools exist in Australia, we sought to develop a screening tool for AYAs based on the original NCCN DTPL, which had reported usefulness in an Asian country, because Japan is more similar in location and ethnicity to Singapore. Furthermore, the ranges for AYAs in the previous studies were 15–25 years (Palmer et al., 2014) and 15–29 years (Patterson et al., 2021b), while those in the study based on the original NCCN DTPL in Singapore was 15–39 years (Chan et al., 2018), which was the same as this study.

In Japan, the Ministry of Health, Labour and Welfare released the Third Basic Plan to Promote Cancer Control Programs in 2018 (Ministry of Health, Labour and Welfare, 2018). It is a policy aimed at enhancing cancer control in the AYA population and promoting the construction of a network of AYAs support teams with multidisciplinary experts. Although the importance of support for AYAs is recognized among healthcare workers, there are differences in the quality and content of support provided by healthcare institutions (Ohara et al., 2018). Furthermore, no screening tools for determining distress and supportive care needs of AYAs have been developed.

This study aims to (i) develop a screening tool for determining distress and supportive care needs of AYAs based on the NCCN DTPL; (ii) evaluate the feasibility, discriminant validity, and test–retest reliability of the tool in clinical settings, and (iii) report prevalence of distress and unmet needs.

Methods

Study design

This study consisted of two phases. The first phase was the development phase, when the Japanese version of the DTPL (DTPL-J) for AYAs was developed. The second phase was the feasibility phase in which its feasibility, discriminant validity, and test–retest reliability were evaluated and AYAs’ prevalence of distress and unmet needs were reported. In the development phase, the following procedures were performed based on a previous study (Inada, 2015): (i) translation of the DTPL from English into Japanese and back translation and (ii) cognitive debriefing using semi-structured interviews with AYAs. Items in the problem list were modified, including deletion of some original DTPL items to better reflect AYAs’ concerns from the interviews. The modified items were reviewed and accepted unanimously by healthcare professionals. In the feasibility phase, the DTPL-J for AYAs was used in a clinical setting for 3 months at the National Cancer Center Hospital (NCCH), Japan.

Procedures

Development phase

This phase was approved by our institutional review board (IRB number: 2018-343). Written informed consent was obtained from all participants.

Translation of the DTPL from English into Japanese and back translation

The DTPL consists of a DT and a problem list of 39 items in five categories: practical problems, family problems, emotional problems, spiritual or religious concerns, and physical problems (Riba et al., 2019).

The DT, which was developed by NCCN to measure cancer patients’ distress, asks patients to rate their level of distress using a scale of 0 (no distress) to 10 (extreme distress). The DT provides a quick measure of distress. It has been validated as a means of distress level assessment in Japanese cancer patients (Akizuki et al., 2005). In the NCCN Guidelines for Distress Management (Riba et al., 2019), a score of ≥4 in a patient with cancer corresponds to clinically significant distress.

Translation of the DTPL into Japanese was approved by NCCN. It was translated by two investigators (H.I. and R.K.). Another investigator (C.H.) who speaks English on a daily basis back translated the Japanese version into English. Three professionals involved in AYAs support on a daily basis (T.H., Y.Y., and E.S.) confirmed that the back translated and original items were synonymous and created a Japanese version of the DTPL as well as a corresponding English version. Finally, we obtained approval from NCCN for the DTPL-J.

Cognitive debriefing

Semi-structured interviews were performed with 40 AYAs at NCCH to determine whether they understood the contents of the items in the DTPL-J. They were also asked whether additional items were needed.

The eligibility criteria were as follows: age between 15 and 39 years and histological diagnosis of malignant neoplasm between the ages of 15 and 39 years. The exclusion criteria were as follows: severe mental symptoms that would have interfered with the investigation; inability to understand Japanese; cognitive functional disorders; or disturbance of consciousness that made it impossible to understand the content of the interview, and physicians’ judgment that participation in the interview would be difficult.

A member of the research team approached potential participants at NCCH to invite them to participate in this study and obtain written informed consent.

The ISPOR task force's report on translation and cultural adaptation recommended a group of 5–8 respondents in the target country (Wild et al., 2005). The age and gender distribution of each group in the sample population was kept equal to reflect the opinions of each age and gender. Sample size was set to 40, with eight participants in each age group (i.e., age 15–19, 20–24, 25–29, 30–34, 35–39 years).

Interviews were conducted by clinical psychologists at NCCH based on the interview guide (Table 1) and recorded. All study participants were provided a prepaid gift card (500 Japanese yen) to thank them for their participation.

Table 1. Interview guide

^a Before the interview, each item was defined based on discussions among healthcare professionals involved in AYA cancer care and AYA cancer survivors.

Two clinical psychologists (Y.Y. and Y.O.) who were engaged in daily medical treatment of AYAs but did not conduct interviews sorted the respondents’ remarks about items that were difficult to understand in Japanese and new items. These remarks were converted into text data as meaningful sentences.

Whether each sentence corresponded to an item that is difficult to understand as a Japanese expression was judged and extracted. Whether each sentence corresponded to additional items was judged and extracted. Three clinical psychologists who are routinely involved in the treatment of AYAs (Y.Y., A.S., and M.T.) organized the concepts of the additional extracted items using the KJ method (Kawakita, 1967).

The extracted items and new items were reviewed by healthcare professionals involved in AYAs care (oncologists, pediatricians, palliative care physicians, psycho-oncologists, nurses, and pharmacists) and AYA cancer survivors.

Revisions were focused on two points: easy to understand the meaning of the Japanese and ability to properly associate each item with what is being asked. Assessment of new items focused on whether they covered problems that AYAs tended to have. The final decision about the modified items and added items was subject to unanimous agreement.

Feasibility phase

This phase was approved by our institutional review board (IRB number: 2019-215). Opt-out information was published on the NCCH website due to the retrospective design.

The draft DTPL-J for AYAs was provided to patients at NCCH. Nurses assessed patients on admission using the draft instrument and consulted with attending physicians or other experts, as they deemed necessary. The medical records of the patients were investigated retrospectively. The eligibility criteria were as follows: (i) age between 15 and 39 years; (ii) histological diagnosis of malignant neoplasm between the ages of 15 and 39 years; and (iii) hospitalization at NCCH at any time from February to April 2020. Some patients underwent multiple screenings because screening was performed at each hospitalization. Response and referral rates to experts before and after the tool was adapted in clinical practice were evaluated to assess the feasibility of the first screening. February–April 2018 was selected as the period before the tool was adapted in clinical practice because no clinical screening for AYAs had been performed during this period. The first and second screenings were used to evaluate test–retest reliability. Sample size was not pre-determined because this was a retrospective study to evaluate the feasibility of the DTPL-J for AYAs.

Data analysis to report prevalence of distress and unmet needs and to evaluate the feasibility, discriminant validity, and test–retest reliability of the DTPL-J for AYAs in clinical settings

Descriptive statistics were calculated for the participants’ demographics, selected items in the problem list, and DT score to report prevalence of distress and unmet needs. Associations between DT score, patient characteristics, and selected items in the problem list were examined using the chi-square test. A DT score of ≥4 was defined as high distress (Huihui et al., 2020). Response and referral rates to experts were evaluated to assess feasibility. Based on a previous study (Chan et al., 2018), we pre-defined a response rate of ≥65% as feasible. The response rate was defined as the proportion of AYAs respondents who were first provided the DTPL-J for AYAs from February to April 2020. Referral rates to experts before and after the tool was adapted in clinical practice were compared using the chi-square test to evaluate feasibility.

Some items, such as dealing with children, child care, ability to have children, dealing with partner, work or school, and housing, were compared with patient demographics, such as whether or not the patient had children or a partner. Some items about social background were selected to evaluate discriminant validity because this information does not change over a short period.

Correlations between two consecutive screenings with a ≥2-week interval were examined to evaluate test–retest reliability using correlation analysis.

Results

Development phase

Participants’ demographic characteristics

Participants’ demographic characteristics are shown in Table 2. Twenty males and 20 females with an average age of 26.7 years were included. Cancer type included bone and soft tissue tumor (n = 21, 52.5%), hematological cancer (n = 7, 17.5%), lung cancer (n = 4, 10%), and other (n = 8, 20%). The most common stage at diagnosis was stage IV (n = 16, 40%), followed by stage I (n = 11, 27.5%).

Table 2. Participants’ demographic characteristics (n = 40)

^a Multiple responses were possible for each participant.

Cognitive debriefing

The participants reported that 34 of 39 (87%) items were understandable. The following five items were difficult to understand: appearance (n = 1, 2.5%); getting around (n = 4, 10%); substance use (n = 2, 5%); insurance or financial (n = 6, 15%); and treatment decisions (n = 9, 22.5%). These items were modified as follows: appearance (how you look, the way that you look), daily activity, use of non-prescription medicine, money (medical expenses, living expenses, insurance), and treatment options.

The item about substance use was modified to refer to the use of non-prescription medicine because substance use is uncommon in Japan (Degenhardt et al., 2019). There was concern that removing substance use would not cover alcohol. However, medical staff at hospitals in Japan routinely gather information about alcohol and smoking use; thus, this change was considered acceptable. On the other hand, the prevalence of non-prescription medicine use such as dietary supplements increased with age among young adults in Japan (Kobayashi et al., 2017). Some supplements, such as St. John's wort and goldenseal, are known to cause clinically important drug interactions and should be avoided by most patients receiving any pharmacologic therapy (Asher et al., 2017). Screening for the use of non-prescription medicine is significant because information about their use is not routinely gathered by medical staff in Japan.

Thirteen items were extracted as additional new items based on interviews. After review by healthcare professionals and AYA cancer survivors, both weight reduction and hair loss were removed. In Japan, many cancer patients experience psychosocial distress from changes in physical appearance, including weight reduction and hair loss (Nozawa et al., 2017; Watanabe et al., 2019). Therefore, it was considered important that not only physical concerns about weight reduction and hair loss but also psychosocial concerns about changes in physical appearance were screened for the item “appearance (how you look, the way that you look).” Supplementary explanations in parentheses had been added to facilitate the extraction of psychosocial concerns. The following 11 items were added: information about illness or treatment; someone to talk to or the consultation environment; important schedule or events; interaction with medical staff; interaction with parents; interaction with other family members; interaction with people other than family members; hospitalization life; mental health of family members; anxiety; and irritation. The draft DTPL-J for AYAs, which consisted of the DT and a problem list of 49 items in five categories (Figure 1), was developed to determine distress and supportive care needs of AYAs in Japan.

Fig. 1. Screening sheet for Japanese adolescents and young adults with cancer.

Feasibility phase

Response rate

Of 251 AYAs, 232 (92.4%) patients were provided the DTPL-J for AYAs at least once. There were 230 (91.6%) patients who underwent at least 1 screening. It was not possible to reach 19 patients. Of the two patients who did not respond, one had delirium and the other had an intellectual disability.

Respondents’ demographic characteristics are shown in Table 3. There were 102 males and 128 females with an average age of 25.7 years. Cancer type included bone and soft tissue tumor (n = 108, 47.0%), hematological cancer (n = 19, 8.3%), germ cell tumor (n = 17, 7.4%), colorectal cancer (n = 15, 6.5%), gynecological cancer (n = 15, 6.5%), and other (n = 56, 24.3%). The most common stage at diagnosis was stage IV (n = 90, 39.1%), followed by stage I (n = 32, 13.9%).

Table 3. Demographic characteristics of the respondents (n = 230)

^a Multiple responses were possible for each respondent.

Referral rates to experts before and after the DTPL-J for AYAs was adapted in clinical practice

Among the patients who responded, 207 (90%) selected at least 1 item and 45 (21.7%) were referred to an attending physician or another expert by their nurse (Table 4). The referral rates to experts other than the attending physician were higher after the tool was adapted in clinical practice (33 of 251, 13.1%) compared with before (15 of 201, 7.5%) (February–April 2018), but this difference was not statistically significant (p = 0.06).

Table 4. Items checked by patients and healthcare workers to whom they were referred

HPS, Hospital Play Staff; MSW, Medical Social Worker.

Distress thermometer score and needs

Based on the DT ≥ 4 cutoff, 69 of 230 patients (30%) had high distress. In the high distress group (n = 69), 66 patients (95.7%) selected at least 1 item. In the low distress group (n = 161), 141 patients (87.6%) selected at least 1 item (p = 0.002).

All items in the problem list were selected at least 3 times. The average number of items selected was 7.0. Anxiety (n = 85, 36.6%) was the most commonly selected item. Regarding the five modified items, the number (proportion) of respondents selecting them were as follows: appearance (how you look, the way that you look), 46 (19.8%); daily activity, 41 (17.7%); use of non-prescription medicine, 3 (1.3%); money (medical expenses, living expenses, insurance), 63 (27.2%); and treatment options, 34 (14.7%). For the 11 new items, the number (proportion) of respondents selecting them were as follows: information about illness or treatment, 51 (22%); someone to talk to or the consultation environment, 13 (5.6%); important schedule or events, 29 (12.5%); interaction with medical staff, 14 (6%); interaction with parents, 17 (7.3%); interaction with other family members, 13 (5.6%); interaction with people other than family members, 20 (8.6%); hospitalization life, 40 (17.2%); mental health of family members, 47 (20.3%); anxiety, 85 (36.6%); and irritation, 31 (13.4%).

Predictive discriminant validity and test–retest reliability

Patients without children did not respond to the item about dealing with children. Patients without spouses or partners did not respond to the item about dealing with partners. Among patients who were not studying or employed, two subjects responded to the item about work or school. Both subjects had worked previously, but were forced to retire due to illness. Patients who were living alone did not respond to the item about housing (Table 5). The items compared were consistent with their social background.

Table 5. Factors related to checked items to verify the validity of the screening tool

^a Worked previously, but was forced to retire due to illness.

Of 230 respondents, 41 (17.8%) responded multiple times due to multiple hospitalizations. These 41 responses from the first and second screenings were included in the analysis of test–retest reliability.

A positive correlation was observed between two consecutive screenings for the following selected items: dealing with children (r = 0.787, p < 0.0001), child care (r = 0.729, p < 0.0001), ability to have children (r = 0.412, p < 0.0001), dealing with partner (r = 0.498, p < 0.0001), work or school (r = 0.641, p < 0.0001), and housing (r = 0.763, p < 0.0001).

Discussion

In this study, we developed the DTPL-J for AYAs as a screening tool to determine distress and supportive care needs based on the NCCH DTPL and evaluated its feasibility, discriminant validity, and test–retest reliability in the clinical setting.

Eleven items, such as the items about illness and treatment information and someone with whom they can talk, were extracted from the interviews. These items reflected the Japanese social and medical issues specific to the AYA population with cancer (Ohara et al., 2018). Someone to talk to or the consultation environment might seem to combine two different ideas into one item. If the item name were “someone to consult or the consultation environment,” AYAs might be hesitant to request a consultation because it is too formal. Thus, the item was described as “someone to talk to or the consultation environment” so that they would feel more comfortable in a consultation.

Some modified or new items were similar to those of the tool developed in Australia (Patterson et al., 2021a). The modified items about daily activity and treatment options were similar to missing doing normal stuff with friends and feeling involved in decision making, respectively. The new items of interaction with parents, interaction with other family members, irritation, interaction with people other family members, important schedule or events, information about illness or treatment, and someone to talk to or consultation environment were similar to mum and/or dad, other family members, anger or frustration, isolation from friends, missing important events, understanding of information, and feeling listened to, respectively.

On the other hand, our modified or new original items included interactions with medical staff, hospitalization life, and mental health of family members. The item about mental health of family members is consistent with previous reported unmet supportive care needs in Japan (Okamura et al., 2021). Most participants in the cognitive debriefing were undergoing treatment for cancer. This might have affected the other two items.

The response rate was over 90%, higher than the 65% we had defined as feasible. The referral rate to multidisciplinary experts was 21.7%, which might have reflected an improvement with the use of the tool compared to the time before its use. This result suggested that the tool could provide an opportunity for a multidisciplinary approach, which is required for the management of AYAs (Zebrack et al., 2010; Warner et al., 2016). Previous studies reported that screening tools are useful for initiating psychosocial and comprehensive care for AYAs (Palmer et al., 2014; Chan et al., 2018). Thus, the tool could lead to activities of AYAs support teams and uniform provision of support for AYAs in Japan. On the other hand, the referral rate to multidisciplinary experts was low in this study. We need to identify the barriers to the use of psychosocial support system such as difficulty scheduling around work/school, cost of care/financial limitations, and lack of knowledge about available resources (Gardner et al., 2014).

The proportion of patients who had high distress (30%) was similar to the proportion in previous studies. The prevalence of distress in AYAs was 20.6, 21.3, and 43.1% in three previous studies (Kim and Yi, 2013; Chan et al., 2018; Michel et al., 2019). On the other hand, a recent international study of AYAs within 3 months of diagnosis reported that 42% experienced distress using a higher cutoff (DT of 5, determined by validation work with AYAs) (Patterson et al., 2021b). Our study might have included AYAs who were diagnosed long ago because the duration after diagnosis was not included in eligibility criteria. The prevalence of distress in this study might be low compared to the prevalence in previous studies because the distress of AYAs decreases as time after diagnosis increases (Chan et al., 2018).

Furthermore, some patients in this study who had supportive care needs reported high distress. This result is consistent with results in a previous study (Riba et al., 2019). Barriers in access to psychosocial care for AYAs are multifactorial. Ensuring standardized referral and repeated introduction of psychosocial care for AYAs is imperative (Holland et al., 2021). However, screening with the tool and intervening faster might help alleviate their distress efficiently and provide continuous comprehensive supportive care.

This study has several limitations. First, the clinical use of the tool was introduced at a single cancer center. Results might not be generalizable to other settings because the NCCH AYAs support system for was robust (Ishiki et al., 2022). The AYA support team consisted of multidisciplinary experts who hold meetings routinely and share the support status of inpatients. In Japan, the number of AYAs per hospital is small and the primary cancer site varies, leading to insufficient staff and resources for experts at each hospital (Ohara et al., 2018). Further studies are required to evaluate the applicability of the DTPL-J for AYAs in other hospitals. Second, referral rates to attending physicians before the tool was adapted in clinical practice were impossible to determine from medical records, because referrals to an attending physician might not have been always recorded. Third, our finding of AYAs’ distress and needs might not reflect that of AYAs in Japan due to selection bias. Finally, this study did not examine the effectiveness of the tool in evaluating the relief of distress. Future studies should also compare DT scores before and after screening to determine the tool's effectiveness in evaluating the relief of distress.

Despite its limitations, the feasibility, discriminant validity, and test–retest reliability of the DTPL-J for AYAs were suggested. The first development of a tool for use with AYAs in Japanese based on supportive care needs can lead to further progress for clinicians and researchers in Japan. Our findings potentially contribute to interventions for distress management among AYAs, AYAs support team activities, and uniform provision of support for AYAs in Japan. We are planning a clinical study to evaluate usefulness of the tool in multiple cancer centers.