Introduction
Palliative care is a preventive, anticipatory care management focused on symptom control that maximizes the quality of life in the final stages of the disease. Palliative approaches replace crisis management with prevention; align treatment decisions with patient and family’s physical, psychological, and cultural needs; and aim to improve the quality of life and lessen the use of invasive end-of-life treatment (Amblàs-Novellas et al. Reference Amblàs-Novellas, Murray and Espaulella2016; Buckley Reference Buckley2008; Dharmarajan et al. Reference Dharmarajan, Wang and Lin2017; World Health Organization 2020). Given these benefits, timely attempts to incorporate palliative principles optimize the quality of life for patients with a life-limiting prognosis such as those with advanced cancer, heart failure, or dementia (Amblàs-Novellas et al. Reference Amblàs-Novellas, Murray and Espaulella2016).
Health-care professionals grapple with the decision about when to begin a palliative approach, and general practitioners find it difficult to maintain palliative identification skills (Hui et al. Reference Hui, Paiva and Del Fabbro2019; Shipman et al. Reference Shipman, Gysels and White2008). Historically, palliative care has been limited to acute terminal diseases with a prognosis of weeks or months. The increasing prevalence of chronic diseases has promoted earlier integration of palliative care into care plans for advanced progressive chronic conditions with a limited life prognosis. When to begin such care plans remains unclear (Hui et al. Reference Hui, Paiva and Del Fabbro2019). Patient indicators that signify an approach to the terminal phase include nutritional, functional, and cognitive decline; elevated overall symptom burden; multimorbidity; and use of resources. The dual concepts of decline and severity attempt to capture the changing picture of a chronically ill patient headed toward death and having a growing need for symptomatic control and a waning need for curative treatments. Identifying patients who can benefit from such a palliative approach requires an understanding of the events preceding the terminal phases of various disease trajectories (Amblàs-Novellas et al. Reference Amblàs-Novellas, Murray and Espaulella2016). Screening tools amalgamating various indications of palliative-care needs have arisen to assist in decision-making.
Research on identifying palliative-care needs for disease-specific states, such as heart failure, prognosticate mortality poorly (Allen et al. Reference Allen, Matlock and Shetterly2019). Conversely, it is also difficult to identify when to begin palliative care for patients with multiple chronic conditions (Arbelle et al. Reference Arbelle, Chodick and Goldstein2014). Recent studies of multimorbid patients have found high symptom burden, poor quality of life, and unmet palliative-care needs (Reinke et al. Reference Reinke, Vig and Tartaglione2019). Proposed reasons for these phenomena include the effect of multiple disease states on the individual, their interaction, and the polypharmacy involved in treating them (Calderón-Larrañaga et al. Reference Calderón-Larrañaga, Poblador-Plou and González-Rubio2012; Nunes et al. Reference Nunes, Flores and Mielke2016).
Multimorbidity, defined as 2 or more co-occurring diseases (Fortin et al. Reference Fortin, Soubhi and Bayliss2007), affects 66% of adults over 65 years old, increases with age in both prevalence and severity (Ofori-Asenso et al. Reference Ofori-Asenso, Chin and Curtis2019), and is associated with increased risk of death. For example, evidence indicates that patients with chronic obstructive pulmonary disease are at increased risk of cardiac diseases such as ischemic heart disease, heart failure, and atrial fibrillation. Treatment modalities for 2 or more conditions prove increasingly clinically complicated, at times negating each other (Roversi et al. Reference Roversi, Fabbri and Sin2016).
Studies show that the accuracy of screening tools varies according to patients’ characteristics (Hui et al. Reference Hui, Paiva and Del Fabbro2019). For example, the National Hospice Organization (1996) guidelines identifying the terminal phase of non-oncologic diseases have been widely accepted in the United States despite their questionable predictive accuracy of 6-month mortality (Fox et al. Reference Fox, Landrum-mcniff and Dawson1999). Other tools commonly used include Scotland’s Supportive and Palliative Care Indicators Tool (Highet et al. Reference Highet, Crawford and Murray2014), England’s Gold Standards Framework Prognostic Indicator Guide (GSF-PIG; King et al. Reference King, Thomas and Martin2005), and Spain’s Necesidades Paliativas [Palliative Needs] tool (v.3.1, 2017) (NECPAL) (Gómez-Batiste et al. Reference Gómez-Batiste, Martínez-Muñoz and Blay2013).
The NECPAL tool was developed by the Qualy Observatory/WHO Collaborating Centre for Palliative Care Public Health Programmes of the Catalan Institute of Oncology in Spain to identify advanced terminal patients in need of palliative-care health and social services. It was translated and culturally adapted from the GSF-PIG for population screening throughout the health-care continuum and for use in a family-centered Mediterranean cultural setting (Gómez-Batiste et al. Reference Gómez-Batiste, Martínez-Muñoz and Blay2013). It incorporates a holistic approach, touching on patients’ suffering and social support and functioning by emphasizing palliative triggers such as frailty, multimorbidity, and geriatric syndromes. Studies have shown content and face validity in Spain (Amblàs-Novellas et al. Reference Amblàs-Novellas, Murray and Espaulella2016; Gómez-Batiste et al. Reference Gómez-Batiste, Martínez-Muñoz and Blay2013). The predictive ability of the original NECPAL (v.1.0, 2011) for mortality at 12 and 24 months was found to have high sensitivity (91.3%, 95% CI: 87.2–94.2%) and negative predictive value (NPV) (91.0%, 95% CI: 86.9–94.0). Previously, the NECPAL was used to classify a person’s imminent need for palliative care based on the number of positive parameters (vs. a dichotomous yes/no for palliative needs) (Calsina-Berna et al. Reference Calsina-Berna, Martínez-Muñoz and Bardes Robles2018). Recently, its creators published the NECPAL 4.0 Prognostic (2021) (Gómez-Batiste et al. Reference Gómez-Batiste, Turrillas and Tebé2020; Turrillas et al. Reference Turrillas, Peñafiel and Tebé2021) containing the surprise question (SQ) and 6 of the 13 original parameters, screening for palliative needs by assigning an end-of-life stage based on how many of the 6 prognostic parameters are identified.
The SQ is a single-item assessment tool meant to screen for people approaching the end of life and asks “Would I be surprised if this patient dies within the next (6, 12, or 24) months?” (Lynn et al. Reference Lynn, Schall and Milne2000). Systematic reviews have found wide degrees of predictive accuracy for 12-month mortality, from less than 15% to over 90% (White et al. Reference White, Kupeli and Vickerstaff2017). The SQ was found to be even less likely to properly discriminate with non-oncologic diagnoses (Downar et al. Reference Downar, Goldman and Pinto2017). In general, the prognostic accuracy of physicians in palliative care among oncologic and non-oncologic diagnoses varies even without specific use of the SQ (White et al. Reference White, Reid and Harris2016). Its use therefore remains relevant only where the patient is well known to the provider and/or as part of a larger multi-item screening tool.
The 13 parameters ask whether there is or has been (1) a request for limitation of therapeutic effort; (2) a provider-identified need for palliative care; (3–5) nutritional, functional, or cognitive decline in the last 6 months unrelated to a reversible process; (6) severe dependence (e.g., bed-bound); (7) multiple recurrent geriatric syndromes (e.g., falls, dysphagia, pressure ulcers, and delirium); (8) multiple persistent symptoms resistant to treatment (e.g., pain, weakness, and digestive issues); (9) severe emotional distress; (10) severe social vulnerability; (11) multimorbidity; (12) increasing use of resources (e.g., at least 3 unplanned admissions in the past 6 months); and (13) disease-specific indicators of severity or progression, titled “The Annex.”
The NECPAL is a complex tool structure comprising multiple subscales of the parameters. For example, functional status can be measured using the Karnofsky Performance Scale (Karnofsky Reference Karnofsky and MacLeod1949), using the Barthel Index (Mahoney and Barthel Reference Mahoney and Barthel1965), or enumerating the declining activities of daily living. Some parameters can be significantly affected by cultural norms (e.g., social vulnerability) or the makeup of health-care delivery (such as readmission rates). The NECPAL therefore requires systematic empirical investigation before adoption into other countries. Its psychometric properties have been assessed after being culturally adapted in Chile (Troncoso et al. Reference Troncoso, Morales-Meyer and Villarroel2021) and Portugal (Santana et al. Reference Santana, Gómez-Batiste and da Silva2020), and it has been used to assess the prevalence of palliative needs among sample populations in other countries (Orzechowski et al. Reference Orzechowski, Galvão and Nunes2019). The original NECPAL has been used here in Israel since 2014 without validation.
The NECPAL’s item-level content validity, inter-rater reliability (IRR), and construct validity via known groups are yet to be studied in the original language or in any translated version to date. No psychometric analysis was found in the assessment of the “Annex” of diseases and their criteria. IRR is of specific importance given the inter-professional nature of palliative care among physicians and nurses when using instruments to identify palliative needs (Myers et al. Reference Myers, Gardiner and Harris2010). The present study undertakes rigorous testing of those additional psychometric properties at both the item and scale levels of the NECPAL that are necessary to determine whether the instrument is appropriate to identify a need for palliative care among geriatric inpatients in Israel.
Methods
This is a two-phase instrument adaptation and validation study.
Phase I: tool formation
Instrument
The NECPAL 2016 v.3.1 was chosen for translation because this version adds more objective frailty measures in an attempt to increase specificity and positive predictive value (PPV; Gómez-Batiste et al. Reference Gómez-Batiste, Amblàs and Costa2017a). It consists of 14 parameters grouped into 4 main categories: I – the SQ; II – demand for palliative care from the patient, caregiver, or provider (parameters 1 and 2); III – general clinical indicators of severity and progression (parameters 3–12); and IV – disease-specific indicators of severity and progression (parameter 13, the “Annex”) (Gómez-Batiste et al. Reference Gómez-Batiste, Amblàs and Costa2017a). The instrument is administered by a health-care provider who either is personally familiar with the patient or has access to the patient’s multidisciplinary (nutritional, functional, cognitive, and/or social) assessments. A patient is considered NECPAL-positive if she/he is positive in category I and one other category. A NECPAL-positive designation implies that a palliative-care approach should be considered. Implications of such an approach include, but are not limited to, discussing advanced directives, do-not-hospitalize orders for patients in end-organ failure, and voluntary refusal of food and fluids and prohibiting curative interventions such as antibiotics and whether to commence life-sustaining treatments such as mechanical ventilation.
Translation and adaptation
The English NECPAL was translated by a native English-speaking registered geriatric nurse with Hebrew proficiency and a native Hebrew-speaking physician with English proficiency working in palliative care. Back-translation was performed by English-speaking professionals with expertise in cross-cultural instrument translation. Original English and back-translated versions were compared for accuracy. Forward- and back-translation, used in similar studies (Lavan et al. Reference Lavan, Gallagher and O’Mahony2018; Santana et al. Reference Santana, Gómez-Batiste and da Silva2020), are part of the Beaton protocol, which is recommended by the World Health Organization (2010).
Content validity
The content-validity assessment package (Lynn Reference Lynn1986) included background information, instructions, a 4-point Likert-scale questionnaire assessing the relevance and clarity of each of the 14 parameters and the disease-specific indicators, and space for qualitative comments. Five expert panelists, working in home care, geriatric hospitals, or academic institutions (2 physicians and 3 nurses with 20 to 40 years of experience in their respective fields), were selected by the researchers for their well-known advocacy and expertise in palliative care, gerontology, or chronic care. Three are certified in palliative care and one in geriatric care. The researchers discussed all first-round comments, adjusted the questions appropriately, and repeated the process a second time. Final feedback was amalgamated into the new, locally adapted, Israeli-NECPAL (I-NECPAL).
Phase II: psychometric testing of the I-NECPAL
Inter-rater reliability
The professionals measuring IRR included 4 physicians and 2 registered nurses working in post-acute geriatric care and certified in either geriatric or palliative care. Together they reviewed 25 medical records of patients at a geriatric medical center in Israel and completed the I-NECPAL tool for each of them. Medical records were randomly chosen for patients discharged in 2019 from rehabilitation, subacute internal medicine, supportive care, frail, or mentally frail units. Power analysis supports a review of 25 records for reaching 80% power, assuming a null hypothesis of kappa to be zero and P ≤ 0.05% (Bujang and Baharum Reference Bujang and Baharum2017; Hong et al. Reference Hong, Choi and Hahn2014; Sim and Wright Reference Sim and Wright2005) in order for a result to be considered meaningful.
Construct validity
Divergent construct validity was assessed via the known-groups method. Two fictitious case studies were written by members of the research team based on experience caring for previous inpatients, one case needing palliative care and one not. Thirty health-care professionals (15 physicians and 15 nurses), almost all certified in at least one specialty (several in more than one: palliative n = 10 [33.3%], geriatric n = 12 [40%], other n = 14 [46.7%]) and working in various settings (community, home care, acute care hospital, post-acute care hospital, and “other”), were asked to complete the I-NECPAL for each case. Power analysis supports a sample size of 30 professionals to reach P ≤ 0.05% and a power of 0.8 to calculate sensitivity and specificity (Faul et al. Reference Faul, Erdfelder and Buchner2009).
Statistical analysis
Content validity
To quantify the content validity of the I-NECPAL, Lynn’s (Reference Lynn1986) suggested stages were used to calculate the content-validity index (CVI) for the individual parameters and for the total scale. Relevance and clarity were rated on a 4-point ordinal scale and grouped (1 and 2 as ratings of agreement; 3 and 4 as disagreement). Each itemized CVI (I-CVI) was adjusted for chance agreement based on the multi-rater kappa (k) statistic adopting the following criteria: Fair = k of 0.40 to 0.59; Good = k of 0.60 to 0.74; and Excellent = k > 0.74 (Cicchetti and Sparrow Reference Cicchetti and Sparrow1981; Fleiss et al. Reference Fleiss, Levin and Paik1981). The overall NECPAL scale-level CVI (S-CVI) was calculated by averaging the kappa values of each I-CVI (Lynn Reference Lynn1986).
Inter-rater reliability
IRR was assessed using Cohen’s kappa, similar to studies of the psychometric properties of other tools for identifying palliative needs (Lavan et al. Reference Lavan, Gallagher and O’Mahony2018). IRR was determined via a percentage agreement similar to an I-CVI but with a multi-rater kappa statistic (Wynd et al. Reference Wynd, Schmidt and Schaefer2003) to analyze the dichotomized data. Criteria for kappa were evaluated using the same guidelines as described above.
Construct validity
A 2 × 2 matrix comparing participants’ decisions on the determination of palliative care using the I-NECPAL was used to measure nonrandom associations and to calculate the sensitivity, specificity, NPV, and PPV (Linsell et al. Reference Linsell, Forbes and Burgess2010). Paired-sample t-tests were performed to test the association between profession (nurse or physician) and certification (palliative or not) on the ability to differentiate significantly the 2 case studies.
Results
Tool formation
Content validity
Table 1 shows the descriptive statistics of the professionals’ ratings for relevance and clarity for each parameter of the translated NECPAL tool in the first and second rounds of expert assessment.
Table 1. Descriptive statistics of content validity of translated tool
Ratings for relevance decreased in the second round for parameters 6 and 13 and neurovascular. Ratings for clarity increased or remained equal in the second round for all parameters except for the SQ, parameters 8 and 9, lung, and neurovascular. The frailty parameter was removed after the second round because of the unanimous view of its redundancy in light of parameters 4 and 6. The final amended tool, the I-NECPAL, was established for testing.
Psychometric testing
Inter-rater reliability
The 25 randomly selected geriatric inpatients (mean age 78.4 ± 8.4 years; male n = 9 [36%], female n = 16 [64%]) were hospitalized in 1 of the 4 inpatient units (subacute n = 6 [24%], rehabilitation n = 6 [24%], supportive care n = 7 [28%], and mentally frail n = 6 [24%]). The analysis of IRR points to good-to-excellent agreement among raters: 10 parameters showed an “excellent” and 4 a “good” level of agreement. Table 2 summarizes the item-level analysis.
Table 2. Inter-rater level of agreement on evaluation of 25 patients using I-NECPAL
Agreement was highest for the SQ among patients in the rehabilitation units (kappa-adjusted CVI = 0.90) and lowest among patients in supportive care (kappa-adjusted CVI = 0.72; Table 3). The groups showed high variability (SD ± 0.16–0.21). Agreement was highest for the remaining parameters among patients in the rehabilitation units (kappa-adjusted CVI = 0.82) and lowest among patients in the dementia care units (kappa-adjusted CVI = 0.72), with lower variability than that for the SQ (SD ± 0.05–0.08).
Table 3. Mean of kappa-adjusted CVI for SQ and I-NECPAL parameters by inpatient department and age group
For the 3 age groups, the highest mean agreement for the SQ was for the oldest (88–93 years) and the lowest was for the youngest (68–71 years). Variability remained high within each age group (SD ± 0.18–0.21). The highest mean agreement of the I-NECPAL tool parameters was for ages 74 to 82 and the lowest was for ages 88 to 93. Variability remained lower for the other parameters than for the SQ.
Construct validity
The results of the 2 × 2 matrix for analyzing the multi-rater (n = 30) dichotomous data are displayed in Table 4. The sensitivity, specificity, PPV, and NPV for the NECPAL were identical to the values calculated for the SQ alone. Use of the NECPAL parameters without the SQ showed decreased sensitivity, increased specificity, and decreased NPV. Use of only those parameters included in the NECPAL 4.0 prognostic gave the best “balance” of values (sensitivity = 0.87, specificity = 0.30, PPV = 0.55, and NPV = 0.69).
Table 4. Measure of sensitivity, specificity, PPV, NPV, and accuracy for each item of the scale in determining known-groups construct validity
a The numeric score was considered accurate for the non-palliative case if it was less than or equal to the intended scoring. The numeric score was considered accurate for the palliative case if it was greater than or equal to the intended scoring.
The paired-sample t-tests portrayed the effect of profession (nurse or physician) and certification (palliative or not) on the ability to differentiate significantly between the 2 case studies using the parameters of the I-NECPAL only, without the SQ. There was a significant difference between the mean score of positive I-NECPAL parameters for the non-palliative case (M = 5.27; t(28) = −2.80, P = 0.01) and the palliative one (M = 8.27; t(28) = −2.80, P = 0.01) when scored by physicians but not when scored by nurses. Furthermore, there was a significant difference between the mean score of positive I-NECPAL parameters for the non-palliative case (M = 5.36; t(27) = −2.82, P = 0.01) and the palliative one (M = 8.27; t(27) = −2.82, P = 0.01) when scored by professionals with a certification in palliative care, regardless of the profession. All differences in means remained significant when only comparing the 6 parameters of the NECPAL 4.0 prognostic (parameters 2, 3, 4, 11, 12, and 13).
Discussion
This is the first study to validate the NECPAL for use in Israel and the first to assess the item-level content validity, IRR, and construct validity of the NECPAL in any language. The results of the psychometric testing of the adapted tool, the I-NECPAL, point to acceptable content and construct validity and the IRR of the I-NECPAL for potential inclusion as a tool for identifying geriatric patients in need of palliative care.
The construct-validity phase revealed how heavily the scale weighs on the SQ, sharing the same PPV, NPV, sensitivity, and specificity values. The geriatric post-acute setting is challenging. Although the cases, presented as inpatient geriatric cases, differ in their palliative needs, most of the practitioners would not have been surprised if either had died within the year. Because the NECPAL requires a positive answer (I would be surprised if this patient died within the next 12 months) for a patient to be considered NECPAL-positive, the currently accepted scoring system may be irrelevant for this population. This is further supported by the results of the IRR analysis, which were highest for the SQ among older patients as well as those in rehabilitation and dementia departments, where the illness trajectory is more uniform. On the other hand, agreement for the youngest group and for those patients in subacute and supportive-care departments, where the illness trajectory is sometimes less clear, was lowest.
This study additionally analyzed the I-NECPAL without the SQ. When considering the objective parameters alone, agreement is excellent for patients in supportive care, rehabilitation, and subacute departments but only good for those in dementia units. When analyzed by age, agreement of the parameters alone was excellent for younger patients but only good in the oldest group. Use of the scale’s objective parameters is less uniformly understood for the longer illness trajectory, that is, dementia but also for the oldest group of patients.
Because this study also measured item-level properties, one can draw inferences about the newer shortened version of the NECPAL 4.0 prognostic by assessing the measures for the relevant items, as this tool shares the SQ as well as parameters 2 through 4 and 11 through 13. The content-validity assessment scored these parameters all as “very relevant/very clear” (scores 3.8–4.0). The IRR was excellent for each of these, except for parameter 2, for which it was “good.” Finally, the measures for sensitivity, PPV, and NPV all increased when focusing on these parameters alone, whereas specificity remained the same. Practically, this means that based on the provided case studies, the prognostic sensitivity of the NECPAL 4.0 portrays the tool’s ability to detect 87% of patients truly in true of palliative care and 30% of patients who do not yet need such care. Of note is that the divergent case studies presented scored 3 and 4 on the relevant NECPAL 4.0 parameters, placing them in the same prognostic stage II with a median of 17.2 months of survival. This may attest only to the ability of palliative certified professionals to significantly differentiate between the cases. Clearly, further study of the implementation of the I-NECPAL in geriatric settings in Israel is warranted.
Limitations of the current study include the type of case studies written for known-groups construct validity. It is possible that these studies were too similar in complexity to non-geriatricians for divergence to be clearly seen using the I-NECPAL, as is evidenced by the lower PPV and NPV. Still, because the tool is meant to also work for subtle cases, this may also be a factor in the weight the SQ holds in the assessment. In addition, there may not have been enough information provided for each case, but sometimes only a little bit of information is available to a health-care professional at the time. Finally, while the IRR phase used a small but homogenous sample of practitioners regarding practice setting, the construct-validity phase used a small and largely heterogenous sample regarding practice setting.
This study’s clinical implications include providing a reliable and validated identification tool for Israeli health-care practitioners in making decisions at pivotal moments along a patient’s life course. Such moments can span weeks or months, are thus difficult to identify, and together form the shift from an all-curative-centered plan of care to one that focuses on alleviating symptoms. These decisions include the culturally sensitive issue of when it is appropriate to discuss advanced directives and the more personally meaningful issue of what defines quality as we approach the end of life. National Health System policies addressing the end-of-life care vary from country to country, reflecting local beliefs and culture as well as evidence-based practice. Practitioners now have not only a validated and reliable I-NECPAL to help with these decisions but an initial evidence base to soundly consider whether to use the SQ based on their own palliative expertise.
Acknowledgments
The authors thank those who participated in this study, which was approved by the Shoham Medical Center institutional review board (IRB) (no. 1.20) and the University of Haifa Faculty of Social Welfare and Health Sciences IRB (no. 2568). Written permission was obtained from Xavier Gómez-Batiste Alentorn, MD, PhD, to translate and test the psychometric properties of the NECPAL into Hebrew.
Funding
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Conflicts of interest
None declared.
