U-TEST, a simple decision support tool for the diagnosis of sarcopenia in orthopaedic patients: the Screening for People Suffering Sarcopenia in Orthopedic cohort of Kobe study (SPSS-OK)

Abstract

We aimed to develop and validate a new simple decision support tool (U-TEST) for diagnosis of sarcopenia in orthopaedic patients. We created seventeen candidate original questions to detect sarcopenia in orthopaedic patients with sarcopenia through expert opinions and a semi-structured interview. To derive a decision support tool, a logistic regression model with backward elimination was applied to select variables from the seventeen questions, age and underweight (BMI < 18·5 kg/m²). Sarcopenia was defined by Asian Working Group for Sarcopenia 2019 criteria. After assigning a score to each selected variable, the sum of scores was calculated. We evaluated the diagnostic performance of the new tool using a logistic regression model. A bootstrap technique was used for internal validation. Among a total of 1334 orthopaedic patients, sixty-five (4·9 %) patients were diagnosed with sarcopenia. We succeeded in developing a ‘U-TEST’ with scores ranging from 0 to 11 consisting of values for BMI (Underweight), age (Elderly) and two original questions (‘I can’t stand up from a chair without supporting myself with my arms’ (Strength) and ‘I feel that my arms and legs are thinner than they were in the past’ (Thin)). The AUC was 0·77 (95 % CI 0·71, 0·83). With the optimal cut-off set at 3 or greater based on Youden’s index, the sensitivity and the specificity were 76·1 and 63·6 %, respectively. In orthopaedic patients, our U-TEST scoring with two questions and two simple clinical variables can help to screen for sarcopenia.

Sarcopenia, defined as age-related decline in muscle mass, muscle strength and physical function⁽¹⁾, is globally regarded as a major problem in an ageing society. Sarcopenia is significantly associated with all-cause mortality among community-dwelling older people⁽²⁾. In orthopaedic patients, sarcopenia has also drawn attention because sarcopenic patients are more likely to experience accelerated loss of muscle mass due to the effect of cytokines⁽³⁾ and decline in physical activity caused by pain⁽⁴⁾.

Despite the well-known significance of sarcopenia, the availability of measurements of muscle mass, muscle strength and walking speed to detect sarcopenia is limited in clinical settings in terms of devices, places and skilled human resources. To palliate this shortage, the SARC-F (Strength, Assistance in walking, Rise from a chair, Climb stairs, and Falls) questionnaire was developed as a simple tool to diagnose sarcopenia rapidly and simply⁽⁵⁾ and has been validated in patients with associated diseases as well as in the general population^(6–9). On the other hand, in our previous study, the diagnostic performance of SARC-F in musculoskeletal disease was shown to be low, having a sensitivity of 41·7 % and a specificity of 68·5 %⁽¹⁰⁾. Diagnostic performance depends on the setting where diagnostic tools are used; to develop a new diagnostic tool that consists of a few simple questions and easily available information will help to detect sarcopenia more efficiently than with SARC-F among orthopaedic patients.

We conducted a large single-centre cross-sectional study to develop and validate a new simple diagnostic tool, ‘U-TEST,’ for sarcopenia in orthopaedic patients.

Methods

This was a large single-centre cross-sectional study, named ‘Screening for People Suffering Sarcopenia in Orthopedic cohort of Kobe study’ (SPSS-OK). The present study followed the guidelines laid down in the Declaration of Helsinki. The study protocol was approved by the local institutional review board (no. 57, 26 January 2017) and the Research Ethics Community of Fukushima Medical University (no. 2850, 28 September 2016), and informed consent was obtained from all patients included in this study. The hospital involved in SPSS-OK, a single-specialty surgical hospital that operated intensively on patients with degenerative diseases, was located in the central part of Kobe City. From August 2016 to January 2020, we recruited patients who were scheduled to undergo total knee or hip arthroplasty or spinal surgery at the time of their visit for preoperative evaluation. Eligible were only those patients who would undergo their first surgery because the implanted artificial materials might potentially interfere with measurement by bioelectrical impedance analysis after surgery. Patients who had neuromuscular disease were also excluded.

Item pooling and questionnaire preparation for the index test

The process from development to validation of the diagnostic tool is shown in Fig. 1. As the first step, four physical and occupational therapists (T. K. and others) individually listed items asking the occupations or activities of daily living which they considered difficult for patients with sarcopenia. As the second step, we conducted a semi-structured interview with four knee osteoarthritis patients with sarcopenia to investigate whether additional important items had been overlooked. We selected seventeen candidate items for the development of the index test. All items were converted to questions with two responses (yes or no) to answer (in Japanese). We then conducted a pilot study to evaluate whether or not the contents of questions were easy to understand and appropriate. Each question was translated into English and then back-translated into Japanese to confirm the conceptual equivalence of the English version. Questions prepared for the index test are shown in online Supplementary Table 1. All these steps were supervised by a psychologist and an internist (T. W. and N. K.) who have experience with development and psychometric testing of questionnaire scales⁽¹¹⁾.

Fig. 1. The process from development to validation of the diagnostic tool.

Definition and measurement of the reference standard

We applied the definition of the Asian Working Group for Sarcopenia (AWGS) 2019⁽¹²⁾ as our reference standard for diagnosis of sarcopenia. AWGS2019 criteria use a combination of low skeletal mass index, and either low handgrip strength or low gait speed. The detailed diagnostic criteria of AWGS2019 are shown in Table 1. We measured appendicular skeletal muscle mass using bioelectrical impedance analysis (MC-780 A; TANITA Co. Ltd). The appendicular skeletal muscle mass index was obtained by dividing appendicular skeletal muscle mass by height squared. Handgrip strength was measured twice for both hands using a grip strength dynamometer (GRIP-D T.K.K. 5401; Takei Scientific Instruments Co. Ltd). For gait speed, the walking time was measured twice on a 10 m straight walkway. Extra 2·5 m walkways for acceleration and deceleration were also constructed. For handgrip strength and gait speed, averaged values were used. All measurements were made by well-trained physical therapists during the pre-surgery visit.

Table 1. Diagnostic criteria for sarcopenia

AWGS, Asian Working Group for Sarcopenia; EWGSOP2, European Working Group on Sarcopenia in Older People 2; IWGS, International Working Group on Sarcopenia; ASMI, appendicular skeletal mass index.

* Measured by bioimpedance analysis.

Measurement of other variables

We also included easily available information such as age (≤69, 70–79, 80 years or older), underweight defined by BMI of <18·5 kg/m²⁽¹³⁾ as candidates for the index test. In addition, data on baseline characteristics were collected as follows: sex, location of surgery (knee, hip or spine) and underlying orthopaedic diseases, and co-morbidities (cancer, chronic lung disease, heart disease, stroke and chronic kidney disease). Heart disease was defined as a history of myocardial infarction, congestive heart failure or angina. Chronic kidney disease was defined as an estimated glomerular filtration rate ≤ 60 ml/min per 1·73 m², calculated using age, serum creatinine level and sex as follows⁽¹⁴⁾: estimated glomerular filtration rate = 194 × serum creatinine^–1·094 × age^–0·287 × 0·739 (if female). Diabetes was defined as a glycosylated Hb value ≥6·5 %⁽¹⁵⁾.

Statistical analysis

We conducted a complete case analysis. For the descriptive analysis, the characteristics of study participants were presented as means and standard deviations for continuous variables and as numbers and proportions for categorical variables.

Development of the diagnostic support tool

For derivation of the diagnostic support tool for sarcopenia, a logistic regression model with backward elimination was applied. The initial model included presence of sarcopenia defined by AWGS2019 criteria as the dependent variable and the seventeen (dichotomous) original items, age (≤69, 70–79, 80 years or older) and BMI (<18·5 or ≥18·5 kg/m²) as independent variables. The criterion for elimination from the model was a P value > 0·05. Because a tool using a points score system is easy to understand and use clinically, we developed a score-based diagnostic support tool⁽¹⁶⁾. We used regression coefficients of selected variables after backward elimination to construct our U-TEST score-based diagnostic support tool^(17,18) . For each selected variable, we divided its regression coefficient by the minimum regression coefficient among them and rounded the answer to an integer value⁽¹⁸⁾. Next, we used the total score of those integer values to create a score chart for use as the diagnostic support tool.

Internal validation of diagnostic support tool

To adjust for optimism following backward selection, we estimated the model’s optimism-corrected performance in accordance with the procedure recommended in the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis Statement⁽¹⁹⁾. Performance was estimated from the AUC of the logistic regression model. Optimism was quantified as the average of differences in AUC of a single logistic regression model fitted for both the bootstrapped resampling data and the original data. The model to be fitted for both data sets consisted of automatically selected variables using backward elimination from the resampling data. Bootstrapping was repeated 200 times. The optimism-corrected performance was calculated by subtracting the effect of optimism from the AUC for the originally selected model.

Test of diagnostic performance

First, we described the prevalence of sarcopenia defined by AWGS2019 criteria according to four categories determined in the order of the total score of the developed support tool (0–2, 3–4, 5–6, 7–11). Second, a logistic regression model including the total score as an independent variable and sarcopenia based on AWGS2019 criteria as a dependent variable was applied to calculate the values for sensitivity, specificity, and positive and negative likelihood ratio using each cut-off. We estimated the optimal cut-off point for the total score at which the sum of sensitivity and specificity becomes maximum based on Youden’s index. Third, the discriminative ability of our developed model was compared with SARC-F using the DeLong test, which measures equality of the AUC-ROC (receiver operating characteristics)⁽²⁰⁾. Finally, to examine whether U-TEST is still more sensitive than SARC-F to predict sarcopenia even when a different definition of sarcopenia is used, we fitted separate logistic regression models including the dependent variables defined as sarcopenia by the European Working Group on Sarcopenia in Older People 2 (EWGSOP2)⁽²¹⁾ and by the International Working Group on Sarcopenia (IWGS)⁽²²⁾. We compared the AUC of the model using U-TEST with that using SARC-F for sarcopenia defined by EWGSOP2 and IWGS criteria, separately. The detailed diagnostic criteria of EWGSOP2 and IWGS are shown in Table 1.

All statistical analyses were done using Stata version 16.1 (Stata Corp.). All tests were two-sided, and P < 0·05 was considered statistically significant.

Since we used registry data (i.e. the SPSS-OK) for the present study, we did not have a pre-determined sample size.

Results

Among 1439 study participants, the 1334 without missing index test or reference standard data were enrolled in the statistical analysis (Fig. 2). Few data were missing on the index test (n 96) or the reference standard (n 9). Table 2 shows the characteristics of study participants grouped by the presence or absence of sarcopenia diagnosed by the AWGS2019 criteria. Their mean age was 69·5 years; 73 % were female. Among them, sixty-five (4·9 %) patients were diagnosed with sarcopenia by the AWGS2019 criteria.

Fig. 2. Study flow chart.

Table 2. Baseline characteristics of study participants

(Mean values and standard deviations; numbers and percentages)

Development and internal validation of the diagnostic support tool

From the result of a logistic regression model with backward elimination, BMI (Underweight), age (Elderly) variables and two questions (Q13 ‘I can’t stand up from a chair without supporting myself with my arms’ (Strength) and Q14 ‘I feel that my arms and legs are thinner than they were in the past’ (Thin)) were selected. The optimism-corrected AUC for the originally selected model was 0·76 (95 % CI 0·69, 0·82). The regression coefficients and the assigned scores for each variable are shown in Table 3. The assigned scores of each variable were calculated by dividing its regression coefficient by that of Q13, which was the smallest, and rounded up to the nearest integer. The outcome of our development efforts was the ‘U-TEST’ whose total score ranges from 0 to 11.

Table 3. Point estimates for significant variables associated with sarcopenia and their assigned scores from a logistic regression with backward elimination model*

(Odds ratios and 95 % confidence intervals)

* The dependent variable was sarcopenia as defined by Asian Working Group for Sarcopenia 2019 criteria, and independent variables were seventeen original questions, age (≤69 (reference), 70–79, ≥80 years) and underweight (BMI < 18·5 kg/m²). The significance level for elimination from the model was P ≥ 0·05.

† The assigned scores were derived by the following process: first, all coefficients were divided by the smallest value of the coefficients in the model (i.e. 0·61). Next, the divided numbers were rounded to integer values (e.g. assigned score for ‘age = 70–79 years’ = 1·07/0·61 = 1·754 ≃ 2).

‡ Q13: ‘I can’t stand up from a chair without supporting myself with my arms.’.

§ Q14: ‘I feel that my arms and legs are thinner than they were in the past.’.

Table 4. Diagnostic performance of U-TEST using different cut-offs

LR+, positive likelihood ratio; LR−, negative likelihood ratio.

Test of diagnostic performance

Fig. 3 shows the prevalence of sarcopenia by AWGS2019 criteria according to U-TEST score categories. The prevalence varied from 1·9 % with scores of 0–2 to 50·0 % with scores of 7–11. Table 3 shows sensitivity, specificity, and positive and negative likelihood ratio at different cut-offs. Based on the Youden’s index, the optimal cut-off point was found to be 3 (sensitivity 76·1 (95 % CI 64·7, 84·7) %, specificity 63·6 (95 % CI 60·9, 66·1) %). On the other hand, with a cut-off of 7 or greater, a greater positive likelihood ratio of 29·3 (95 % CI 10·7, 79·9) was obtained (sensitivity 13·4 (95 % CI 6·3, 24·0) %, specificity 99·5 (95 % CI 99·0, 99·8) %).

Fig. 3. Prevalence of sarcopenia by U-TEST score.

Comparison of discriminative ability of U-TEST with SARC-F

Fig. 4 shows the receiver operating characteristics curve of U-TEST and SARC-F to identify sarcopenia. The AUC of U-TEST and SARC-F were 0·77 (95 % CI 0·71, 0·83) and 0·57 (95 % CI 0·50, 0·64), respectively, and the difference between them was statistically significant (P < 0·001).

Fig. 4. Comparison of receiver operating characteristics (ROC) curves between U-TEST and SARC-F (Strength, Assistance in walking, Rise from a chair, Climb stairs, and Falls). (), ROC curve for U-TEST; (), ROC curve for SARC-F.

Discriminative ability of U-TEST for sarcopenia defined by European Working Group on Sarcopenia in Older People 2 and International Working Group on Sarcopenia criteria

The prevalence of sarcopenia by EWGSOP2 and IWGS was 3·7 % (50/1334) and 3·2 % (43/1334), respectively. The AUC of U-TEST for sarcopenia defined by EWGSOP2 and IWGS criteria were 0·80 (95 % CI 0·69, 0·91) and 0·73 (95 % CI 0·65, 0·82), respectively. Both AUC were greater than those of SARC-F (0·57 (95 % CI 0·43, 0·72) for EWGSOP2 and 0·61 (95 % CI 0·52, 0·69) for IWGS).

Discussion

We developed a new diagnostic tool for sarcopenia in patients with orthopaedic disease (U-TEST) that consists of only two questions and two simple clinical variables (older age and underweight defined by BMI). We found that its diagnostic performance was high enough for clinical use, and acceptable even for diagnosing sarcopenia by the EWGSOP2 and IWGS definitions broadly used worldwide. Given the inadequacy of performance of SARC-F in patients with orthopaedic disease⁽¹⁰⁾, we believe that our new tool can replace SARC-F to screen sarcopenia in these patients.

Of the two original questions selected, Q13 ‘I can’t stand up from a chair without supporting myself with my arms’ can be useful as a substitute for measurements of handgrip strength and gait speed. Similarly, Q14 ‘I feel that my arms and legs are thinner than they were in the past’ and underweight (BMI < 18·5 kg/m²) can detect a decline in muscle mass, which means these questions can work as an alternative to bioelectrical impedance analysis. Taking these facts into consideration, the combined use of the selected variables is sufficiently clinically valid to screen sarcopenia with high discriminative ability, considering the operationalisation in AWGS2019 criteria of sarcopenia based on low muscle mass and low muscle strength or physical function.

Several previous studies have developed simple methods to diagnose sarcopenia with excellent performance. Typically, they attempt to improve SARC-F by adding body measurements to questions. SARC-CalF, developed in Brazil, adds calf circumference to SARC-F and is reported especially to improve sensitivity compared with SARC-F^(23–25). A study in Indonesia evaluated performance when the thigh circumference was added to SARC-CalF, and specificity was again shown to be improved⁽²⁶⁾. Compared with these methods, our U-TEST seems to have comparable detectabilities with fewer simple questions, combined with routine measurements (BMI and age) that are acceptable in clinical practice. A research group in Italy also developed a sarcopenia risk assessment tool for community-dwelling elderly that consists of only five (or seven) questions without any body measurement⁽²⁷⁾. These tools can replace SARC-F for screening sarcopenia because their high sensitivity and low negative likelihood ratio are much better than those of SARC-F⁽²⁸⁾. Considering the high values of specificity and positive likelihood ratio (>10) when the cut-off is set at 7 or greater, our tool is also expected to be a useful option especially when confirming the diagnosis of orthopaedic patients with sarcopenia^(29,30) . Conversely, in this case (when the cut-off is set at 7 or greater), it is not useful for screening for sarcopenia because of its low sensitivity and negative likelihood ratio.

Our study has several limitations. First, the measurement of muscle mass was conducted via bioelectrical impedance analysis, whereas the use of dual-energy X-ray absorptiometry or computed tomography is widely recommended in the application of most criteria^{(21,22,31–33)} . However, EWGSOP2 and AWGS2019 also recommend use of bioelectrical impedance analysis and suggest a cut-off for appendicular skeletal muscle mass in the case of bioelectrical impedance analysis^(12,21) . We believe that bioelectrical impedance analysis is a better option with respect to feasibility in the clinical setting. Second, the generalisability of our results for orthopaedic patients could be limited because this is a single-centre study. In future studies, external validation should also be evaluated. Third, we were unable to compare the AUC of the U-TEST to that of SARC-CalF, which includes the calf circumstance among the body measurements because we did not measure the calf circumference. Fourth, we lack external validation data. Therefore, although the optimism inherent in backward variable elimination was addressed by internal validation, we should interpret carefully the results of U-TEST’s diagnostic performance, which might be overly optimistic. Finally, the unexpectedly low prevalence of sarcopenia in orthopaedic patients may be due to selection bias. Patients with orthopaedic disease who are physically compromised to the extent that surgical intervention is not indicated for the disease are more likely to have sarcopenia. However, such patients were unlikely to be included in the current study, which was conducted at a single-specialty (degenerative joint disease) surgical hospital.

In conclusion, we developed a new diagnostic tool (U-TEST) for sarcopenia in orthopaedic patients and conducted its internal validation. Two simple questions combined with such easily available clinical information as age and BMI are sufficient to screen sarcopenia easily without consuming time and manpower. Considering the importance of sarcopenia in orthopaedic patients, U-TEST is a useful measure that facilitates screening for sarcopenia among these patients.