Psychometric validation and cross-cultural adaptation of the Integrated Palliative care Outcome Scale in Polish (IPOS-Pol)

Abstract

Objective

The study aimed to assess the reliability and validity of the IPOS-Pol for patient self-reporting.

Method

Patients (>18 years of age) with advanced cancer admitted to three palliative care centers (inpatient units and home-based) were recruited to a multicenter, cross-sectional, observational, prospective study. Participants provided responses to the IPOS-Pol Patient version and the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire – Core 15 – Palliative Care (EORTC QLQ-C15-PAL) Polish version at baseline (T1) and four to seven days later (T2). We assessed test–retest reliability, internal consistency, and construct validity of the tool.

Results

One hundred and eighty patients were included. Test–retest reliability demonstrated no statistically significant differences in the average outcomes of the IPOS-Pol between T1 and T2 (27.2 ± 9.2 vs. 26.5 ± 8.7; p > 0.05). The intra-class correlation coefficient between T1 and T2 was r = 0.83 (p < 0.0001), the intra-class correlation coefficient for test–retest reliability of the IPOS-Pol items ranged from 0.63 to 0.84 (p < 0.0001), and the Cronbach's α coefficient for internal consistency was 0.773. The correlation coefficient between the IPOS-Pol and EORTC QLQ-C15-PAL total score was 0.79 (p < 0.001).

Significance of results

The patient version of the Polish adaptation of IPOS is a valid and reliable outcome measure for assessing symptoms and concerns of individuals receiving palliative care, as well as the quality of care provided.

Introduction

Patient-reported outcome measures can provide insights into subjective matters known only to the patient with regard to his/her quality of life (QoL), which is the primary interest of palliative care (PC; Hearn and Higginson, 1999; Sepúlveda et al., 2002; Antunes et al., 2014; Collins et al., 2015). There are several tools that are recommended for monitoring QoL dimensions in PC patients in Poland, some of which are validated (Leppert et al., 2014).

The Integrated Palliative care Outcome Scale (IPOS) combines the most important items from the Palliative care Outcome Scale (POS) with items from the symptom version (POS-S), rendering it brief and simple for use with patients with advanced diseases (Hearn and Higginson, 1999; Schildmann et al., 2016). It comprehensively evaluates common physical and emotional symptoms, information needs, practical concerns, family and friends’ anxieties, the ability to share feelings with family and friends, and the overall feeling of being at peace during the preceding three or seven days. The tool is validated in English (Murtagh et al., 2019) and several other languages (Sakurai et al., 2019; Sterie et al., 2019; Hocaoglu et al., 2020; Vlckova et al., 2020; https://pos-pal.org/maix/translations.php Accessed January 27, 2020). It can be used both for research and in clinical settings, irrespective of their place of care (hospital, hospice, nursing home, or home care). A version for patients is used as a self-report outcome measure which can be completed with help from a relative, a friend, or a staff member. Staff-proxy report version can be used when self-administration is not possible or alongside patient completion IPOS staff version. It can be used as a comparison between the patient and staff assessment. Although this scale is used increasingly in several countries (Schildmann et al., 2016; Beck et al., 2017; Sakurai et al., 2019; Sterie et al., 2019; Veronese et al., 2019; Antunes and Ferreira, 2020; Hocaoglu et al., 2020; Vlckova et al., 2020), it is less well known in Poland and has thus far rarely been applied in practice.

The study aimed to assess the reliability and validity of the IPOS Patient version in Polish (IPOS-Pol).

Methods

Adult (>18 years of age) advanced cancer patients who spoke Polish and were hospitalized in St Lazarus Hospice in Krakow, Hospice Palium at the Chair and Department of Palliative Medicine (Karol Marcinkowski University of Medical Sciences in Poznan), or the Sue Ryder Home in Bydgoszcz were recruited to this multicenter, cross-sectional, observational prospective study between May 2017 and May 2019. The PC settings included PC inpatient units and home-based PC services.

The inclusion criteria were: being over 18 years of age; having a diagnosis of advanced cancer in stage B (advanced unstable disease with a prognosis of a few months of life) or C (deteriorating condition with exacerbations and a prognosis of a few weeks of life) according to Gold Standards Framework (GSF) needs-based coding (The GSF Prognostic Indicator Guidance, 2019); being in stable condition within the study period and being able to complete the IPOS-Pol questionnaire unaided (as judged by the PC team) or with very little help; and having given informed consent to the study. Patients were recruited by the staff of the hospices (psychologists, nurses, physicians) participating in the study. Patients who were assessed by the team not to be able to complete the survey (cognitively impaired or too unwell) were excluded.

The IPOS-Pol Patient version (with a one-week recall period) and the Polish version of the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire – Core 15 – Palliative Care (EORTC QLQ-C15-PAL) in patients with advanced cancer (Groenvold et al., 2006; Leppert and Majkowicz, 2013) were completed by patients at two time points: between day 7 and day 10 after admission to PC (first survey, T1), and then between day 4 and day 7 after the completion of the first survey (second time point, T2). At both time points, patients were asked if they wish a member of the staff to be present or help them when they were completing the scales.

Patient characteristics, including age, gender, cancer type, disease stage (based on the GSF) and performance status [(according to the Eastern Cooperative Oncology Group Performance Status (ECOG PS) score] were obtained from medical records or evaluated by the participating staff during the survey.

IPOS, Polish version

The Polish version of IPOS was translated and adapted at St Lazarus Hospice in Krakow, with reference to the Palliative care Outcome Score (POS) Manual for Translation, Cross-Cultural Adaptation and Psychometric Testing (Antunes et al., 2015). The process was coordinated by the POS development team from the Cicely Saunders Institute of Palliative Care, Policy and Rehabilitation, King's College, London, UK. Forward/backward blind translations by native speakers and medical staff experienced in PC were performed with two expert reviews after each translation.

The scale consists of 10 questions with 17 items. The first question is open-ended and refers to the patient's perception of his/her main problems or concerns experienced during the preceding week. The next eight questions are aimed at registering the patient's most important concerns with regard to symptoms (pain, shortness of breath, weakness or lack of energy, nausea, vomiting, poor appetite, constipation, sore or dry mouth, drowsiness, poor mobility, other symptoms), feeling anxious or worried about the illness or treatment, anxiety and worries of family members and friends, feeling depressed, general sense of peace, ability to share feelings with family and friends, need for information, and practical problems resulting from illness (financial or personal). The answers are rated in a scale from 0 (best = not affected) to 4 (worst = most affected). The last item indicates whether the patient completed the IPOS-Pol questionnaire on his/her own or with help from friend, relative, or a member of staff (www.pos-pal.org).

Following recommendations by the authors of POS (https://pos-pal.org/maix/how-to-score.php. Accessed September 26, 2019), the scale should be interpreted individually for each patient at a specific point in time, which is why there are no categories that classify the overall assessment. In order to compare the outcomes between time points, the total IPOS score can be calculated and can range from 0 to 68 (the sum of 17 items scored at 0–4 for each). Question 1 (an open-ended question asking the patient about his/her main problems or concerns), the section of Question 2 relating to other symptoms, and Question 10 (regarding whether the patient completed the questionnaire on his/her own or with help) are not included in the overall IPOS score. The higher the score, the greater the symptom burden and the unsatisfied needs of the patient.

EORTC QLQ-C15-PAL, Polish version

EORTC QLQ-C15-PAL is an abbreviated version of the EORTC QLQ-C30 developed for PC. A validated Polish version of this questionnaire (Leppert and Majkowicz, 2013) was used for criterion validity analysis in the present study. EORTC QLQ-C15-PAL consists of 15 items: 3 functional (walking difficulties, need to stay in bed/a chair during the day, help needed with daily living), 11 assessing symptoms (shortness of breath, pain, trouble sleeping, feeling weak, lack of appetite, nausea, constipation, feeling tired, pain interference with daily activities, feeling tense, feeling depressed; rated on a verbal 4-point scale: from 1 = “not at all” to 4 = “very much”), and a final item relating to overall quality of life (measured on a 1–7 scale from “very poor” to “excellent”). The EORTC QLQ-C15-PAL score ranges from 14 to 56 (14 items rated between 1 and 4, with Question 15 excluded from the score). Similar to IPOS, this scale has no defined score ranges with respect to a general assessment and the higher the score, the greater the symptom burden.

Feedback from patients and staff involved in the study

Respondents in case of any doubts relating to IPOS could leave their feedback by completing a short questionnaire regarding problems with answering or understanding any word or item. They were also asked about any suggested item changes with the request of further explanations or comments. The last question was intended for comments from the staff assisting the patients in completing the questionnaire.

Statistical analysis

Descriptive statistics with means and standard deviations or medians and quartiles where appropriate were used. We tested reliability including test–retest reliability and internal consistency. The t-test for dependent samples or a Wilcoxon test was used to check the test–retest reliability. Additionally, intra-class correlations coefficient between IPOS-Pol items using the first and second measurements was measured. The level of significance for checking the repeatability was assumed to be α = 0.003 (Bonferroni correction for multiple comparisons). Internal consistency of the IPOS-Pol was measured with the application of Cronbach's α coefficient. We considered Cronbach's α coefficient acceptable level of ≥0.7 (Taber, 2018). Construct validity was evaluated using Spearman's correlation between corresponding items of IPOS-Pol and EORTC QLQ-C15-PAL. The correlation coefficients of ≥0.5 were considered as large (Taber, 2018). Differences between particular groups of item variables were checked with the application of Mann–Whitney U test. The level of significance was assumed to be p < 0.05. Statistical analysis was conducted using Statistica 13.3 (TIBCO Software, Inc.).

The study was conducted in compliance with the Declaration of Helsinki and the study protocol was approved by the Jagiellonian University Bioethics Committee (No. 122.6120.71.2017, March 30, 2017).

Results

One hundred and eighty patients with advanced cancer receiving palliative care completed the study. Ninety one (50.6%) of the group preferred to complete the questionnaire in the presence of an interviewer and with their aid. Thirty five (19.4%) completed the scale with an aid of the relative. Twenty individuals were excluded from the study due to their deteriorating health condition and inability to complete the questionnaire at the second time point (T2). Eight patients declined participation. Demographic data and clinical diagnosis are presented in Table 1.

Table 1. Patient characteristics

ECOG PS, Eastern Cooperative Oncology Group Performance Status; GSF, Gold Standards Framework prognostic indicator; n, number of participants; SD, standard deviation.

Test–retest reliability demonstrated no statistically significant differences in average outcomes of the IPOS-Pol between T1 and T2 (mean 27.2 ± 9.2 vs. 26.5 ± 8.7; p > 0.05). In addition, test–retest reliability calculated by intra-class correlation coefficient between T1 and T2 was r = 0.83 (p < 0.0001) and between the IPOS-Pol items were in the range 0.63–0.84 (p < 0.0001). Statistically significant differences were found only in 1 out of the 17 items (pain). The test–retest reliability of the IPOS-Pol items is presented in Table 2.

Table 2. Test–retest IPOS-Pol reliability

^a Wilcoxon test.

^b t-test.

IQ,  quartile deviation; SD, standard deviation. Values in bold represent statistically significant differences.

The internal consistency of the IPOS-Pol measured using Cronbach's α coefficient was 0.773. Only Question 7 (related to sharing feelings) had a slightly negative impact on the internal consistency of the test and, when this was removed from analysis, the consistency increased to 0.789 (Table 3).

Table 3. Internal consistency of the IPOS-Pol

The Spearman correlation coefficient between the IPOS-Pol and the Polish version of the EORTC QLQ-C15-PAL results was 0.79 (p < 0.001; Figure 1). The correlation coefficients between IPOS-Pol and Polish version of EORTC QLQ-C15 items are shown in Table 4.

Fig. 1. Correlations between IPOS-Pol and EORTC QLQ-C15-PAL (Polish version).

Table 4. IPOS-Pol criterion validity according to EORTC QLQ-C15-PAL (Spearman's rho)

* Pearson correlation.

Patients with a shorter prognosis (GSF stage C) reported significantly higher total IPOS scores (31 vs. 24 points in patients GSF stage B). Scores for some IPOS items (weakness, poor appetite, sore or dry mouth, drowsiness, and poor mobility) were higher in patients with a shorter prognosis (GSF stage C) and in patients with a lower performance status (ECOG 3–4) (Table 5).

Table 5. Comparisons within the IPOS-Pol studied-group

ECOG PS, Eastern Cooperative Oncology Group Performance Status; GSF, Gold Standards Framework prognostic indicator; IQ, quartile deviation. Values in bold represent statistically significant differences.

Twenty-two (12.2%) patients gave their comments and suggestions in the feedback form. Thirty patients (16.7%) reported problems with answering at least one of the questions. The most common item reported by these patients was Question 7 (“Have you been able to share how you are feeling with your family or friends as much as you wanted?”; n = 8; 4.4%) and Question 9 [“Have any practical problems resulting from your illness been addressed (such as financial or personal)?”(n = 8; 4.4%)]. According to the staff participants, the source of the problem with Question 7 was not a lack of understanding, but originated from individual difficulties with communication and sharing feelings with relatives and friends. None of the respondents claimed that he/she had found Question 7 irritating or worrying; however, the staff commented that in three cases (1.7%), the question caused considerable discomfort for patients. In relation to Question 9, a need for clarification was indicated (“Does the question refer to family or health care providers?”). Other questions that some patients found difficult to answer were Question 4 (“Have any of your family or friends been anxious or worried about you?”; n = 7; 3.9%) and Question 8 (“Have you had as much information as you wanted?”; n = 5; 2.8%). In the case of Question 4, difficulties with answering resulted from poor communication within the family or with friends, rather than not understanding the question (“How could I know it?”; “It's hard to say, I know nothing about it”; “It's hard to say, they have it in their hearts”). Difficulties with answering Question 8 resulted from differing expectations and needs relating to getting information. Patients claimed that the question was unclear (“what information?”; “information from whom?”) and suggested that, in their opinion, it should be re-edited to make it more clear. Of the remaining questions only single respondents reported difficulty answering. The IPOS-Pol questionnaire does not contain any vocabulary that the patients found difficult to understand. The only request for clarification regarding vocabulary came from one person, who asked for an explanation of the term “moderately.”

The main advantage of the IPOS-Pol pointed out by all members of the participating staff was that it allowed patients to establish their hierarchy of problems at the beginning (Question 1) and report symptoms not listed elsewhere, which, according to the staff participants, increased respondents’ sense of security.

Discussion

Although no ideal tool exists, the use of IPOS as a basis for conversation may facilitate discussions about care needs and could be used in palliative care to enable the best possible person-centered care (Högberg et al., 2019). The IPOS being a patient-reported outcome measure integrating POS and POS-S is a comprehensive tool that was primarily developed to be used with patients who have advanced cancer, but it could also be successfully implemented in patients with diseases other than cancer (Antunes et al., 2015). As for now illness-specific versions of IPOS for people with renal diseases, dementia and other long-term neurological conditions have been implemented (www.pos-pal.org). This outcome measure has so far been translated into 13 languages and more translations are currently being undertaken and new studies are being conducted (Lind et al., 2018; Sakurai et al., 2019; Sandham et al., 2019; Sterie et al., 2019; Veronese et al., 2019; Antunes and Ferreira, 2020; Hocaoglu et al., 2020; Vlckova et al., 2020).

The present study demonstrates that the IPOS-Pol is a reliable and valid tool for assessing and monitoring the subjective symptoms and concerns of Polish-speaking patients with advanced cancer. Good reliability of this scale was confirmed in the test–retest analysis (r = 0.83; p < 0.0001), which is consistent with other studies (Murtagh et al., 2019; Sakurai et al., 2019; Sandham et al., 2019; Sterie et al., 2019; Antunes and Ferreira, 2020; Vlckova K. et al. 2020). A significant difference was found only in 1 out of 17 items (related to pain), which was probably due to dynamic changes of symptom burden within this group of respondents. In concordance with this finding, we indicated significant differences between patients with longer and shorter prognosis and patients with higher or lower performance status as determined by GSF and ECOG PS, respectively. Higher scores reflect more symptoms and concerns in patients with more advanced diseases. Other authors have indicated IPOS responsiveness to change in time (Murtagh et al., 2019; Sterie et al., 2019).

A high internal consistency of the IPOS-Pol was confirmed. A negative impact of Question 7 (sharing feelings with family and friends) on the internal consistency of the whole scale (α = 0.789 vs. α = 0.773) was small. This item was an important issue for most of the respondents, who revealed communication difficulties.

The present study confirmed the criterion validity of the IPOS-Pol measure by correlating its outcomes with those of another standardized tool: the Polish version of the EORTC QLQ-C15-PAL scale, which had undergone language adaptation and validation. Good validity of the IPOS was also demonstrated by other authors (Murtagh et al., 2019; Sakurai et al., 2019; Sterie et al., 2019; Antunes and Ferreira, 2020). In earlier studies assessing other language versions, the IPOS was demonstrated to be easy to comprehend, acceptable, and not burdensome for patients or staff to complete (Schildmann et al., 2016; Veronese et al., 2019). The feedback from the patients involved in the present study also indicates that the questionnaire is feasible, understandable, and fully acceptable to patients. Only some patients needed clarification regarding Questions 8 and 9 (<5%). Both questions refer to the patient's family and friends, as well as to the medical staff, and are aimed at identifying the areas in which the patient's need for information is unsatisfied and which practical problems the patient worries about, regardless of who should provide the information or help address the problems. Having staff members present while patients complete the IPOS maximizes the advantages of this tool in helping to screen problems to be addressed in order to improve the quality of patient care. Appropriate education for staff members prior to the implementation of IPOS is also of great importance (Högberg et al., 2019; Murtagh et al., 2019).

Possible limitations to the study were that it was restricted to cancer patients with a short prognosis (patients in GSF stage A, who are not often admitted to PC services, were not included). IPOS is dedicated to PC patients with malignant and nonmalignant diseases. We chose, however, to limit our group to cancer patients because the vast majority of patients in PC in Poland (>90%) are individuals with cancer disease. Patients with noncancer diagnoses are usually admitted to hospice care in the terminal stage of the diseases when they are too unwell to participate in a study. We decided to limit our study to the patient version of IPOS, which can be used with an aid of the staff professionals, because this version is currently preferred more often in practice than the staff version. The another weakness of the study is the lack of inter-rater comparison.

In conclusion, the present study has confirmed the reliability and criterion validity of the Polish adaptation of the IPOS Patient version. The scale can be recommended as a useful and easy-to-implement outcome measure to assess the symptoms and concerns of patients with advanced cancer receiving palliative care, as well as the quality of the care provided.