Background
In residential long-term care (LTC) settings, falls are a common cause of injuries and can even lead to death, particularly among the very frail elderly. In Canada, more than one third of seniors will experience a fall, and it is the leading cause of injury to them (Accreditation Canada, 2014). Severe injuries (e.g., hip fractures) can lead to lengthy and costly hospital stays and the need for extensive rehabilitation interventions. Direct health care costs for falls in Canada are estimated at $2 billion annually (Accreditation Canada, 2014). As Canada’s population ages and the continuing care sector focuses on keeping older persons at home longer with the support of home care, LTC homes can expect to see the frailty level of residents continue to increase (Canadian Institute for Health Information, 2011), which could put additional strain on existing resources and impact outcomes.
In response to these risks, falls are an important quality indicator in LTC settings (Frijters, van der Roest, Carpenter, & Finne-Soveri, Reference Frijters, van der Roest, Carpenter and Finne-Soveri2013; Jones et al., Reference Jones, Hirdes, Poss, Kelly, Berg and Fries2010; Poss & Hirdes, Reference Poss and Hirdes2016) and are generally defined in standard assessments as an unintentional change in position (Morris et al., Reference Morris, Berg, Bjorkgren, Finne-Soveri, Fries and Frijters2010). Naturally, these incidents will range from very minor events leading to no injury or contact, to a major fall resulting in serious injury or death. Various risk management approaches are used in LTC settings, such as attempting to reduce all falls or to reduce mainly injurious falls, and focusing on near-miss reporting as a means to identify and prevent risk (Wagner, Scott, & Silver, Reference Wagner, Scott and Silver2011). It is well established that a recent history of falls is the leading factor in predicting a future fall (Barker, Nitz, Choy, & Haines, Reference Barker, Nitz, Choy and Haines2009; Kiely, Kiel, Burrows, & Lipsitz, Reference Kiely, Kiel, Burrows and Lipsitz1998). Once older adults fall, they are more likely to fall again, particularly as their frailty increases (Barker et al., Reference Barker, Nitz, Choy and Haines2009; Poss, n.d.; Poss & Hirdes, Reference Poss and Hirdes2016). Additional studies have assessed risk factors among elderly populations who have not experienced a fall to determine if there are common risk factors, and many of these are reflected in the more prominent tools being used today (Cameron, Mustard, & Mayer, Reference Cameron, Mustard and Mayer1997; Fonad, Wahlin, Winblad, Emami, & Sandmark, Reference Fonad, Wahlin, Winblad, Emami and Sandmark2008; Kiely et al., Reference Kiely, Kiel, Burrows and Lipsitz1998; Sterke, Verhagen, Van Beeck, & van der Cammen, Reference Sterke, Verhagen, Van Beeck and van der Cammen2008; Tinetti, Reference Tinetti1987; Vu, Weintraub, & Rubenstein, Reference Vu, Weintraub and Rubenstein2004).
Completing regular and accurate assessments and the associated care planning that follows, is an important time investment for clinical staff that has a strong potential to influence quality outcomes for LTC residents (Fitzgerald et al., Reference Fitzgerald, Hadjistavropoulos, Williams, Lix, Zahir and Alfano2016; Miedany, El Gaafary, Toth, Palmer, & Ahmed, Reference Miedany, El Gaafary, Toth, Palmer and Ahmed2011). Despite the availability of various risk and screening tools for clinicians preparing care plans, it remains unclear which tool is most effective at predicting falls. The lack of standardization can result in clinicians using several tools to “cover all bases” or using outdated ones, resulting in increased effort on their part, with potentially little benefit to the resident. The scarcity of clinical resources in LTC is approaching a critical level in Canada, so any work that is not adding direct value deserves a close review (Canadian Institute for Health Information, 2016b).
The objective of this study is to evaluate the predictive accuracy of three separate fall risk assessment (FRA) tools; the interRAI Falls Clinical Assessment Protocol (CAP), derived from the LTC Facility (LTCF) or Minimum Data Set (MDS) 2.0 assessments; the Scott Fall Risk Screen (SFRS); and a modified Fall Risk Tool that was implemented as part of a provincial Fall Reduction Strategy in Nova Scotia. Conclusions regarding the effectiveness of these commonly used tools in Canadian LTC homes can be used to guide evidence-based practice.
FRAs
Although falls pose a substantial risk to the health and quality of life of older adults, they are a common occurrence, particularly in the LTC setting. It is estimated that 15–20 per cent of Canadian LTC residents experience a fall each quarter (Hirdes, Mitchell, Maxwell, & White, Reference Hirdes, Mitchell, Maxwell and White2011). Clinicians respond to this risk by screening new and existing residents as a typical part of health care providers’ fall prevention programs. Screening residents is intended to help clinicians to develop interventions to prevent these incidents from occurring or to minimize injury if they do. These interventions prioritize residents at higher than normal risk with therapies, treatments, or preventive measures that go beyond the universal steps employed to prevent falls among all residents in a nursing home setting (e.g., nurse call bell system, grab bars, clear pathways).
Theoretically, a fall risk “screen” would provide a quick and simple evaluation of the likelihood that a resident will experience a fall. From there, a resident at a high level of risk would undergo a more comprehensive, multifactorial, and interdisciplinary fall risk “assessment” that would provide the clinician with more insight into specific risk factors and potential interventions to put in place.
Several FRA tools have been developed, tested, and applied across various populations, settings, and regions. A systematic review of 20 different FRAs revealed that the majority of them were developed specifically for elderly populations and were mainly in hospital or LTC home settings. Although the various tools generally assessed similar characteristics, there was wide variability both in the amount of time the tools took to complete and the predictive accuracy of their results (Perell et al., Reference Perell, Nelson, Goldman, Luther, Prieto-Lewis and Rubenstein2001).
Most FRAs in use today include intrinsic risk factors such as a history of falling, medical conditions (e.g., incontinence, orthostatic hypotension), problematic drug utilization (e.g., polypharmacy), assistive device use (e.g., cane, walker) and gait issues. Extrinsic factors (e.g., environmental factors that pose a slip or trip hazard) are usually addressed through facility maintenance and architectural design. Falls affected by these intrinsic and extrinsic factors might be more easily anticipated and prevented than those related to unexpected physiological changes (e.g., stroke). The latter causes are more difficult to avoid because the resident would not typically be screened as having elevated fall risk upon assessment (Dykes, Reference Dykes2017).
Common FRA tools used in North American hospitals and LTC settings include the Morse Fall Scale (Morse, Morse, & Tylko, Reference Morse, Morse and Tylko1989), St. Thomas's Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY) (Latt, Loh, Ge, & Hepworth, Reference Latt, Loh, Ge and Hepworth2016; Oliver, Britton, Seed, Martin, & Happer, Reference Oliver, Britton, Seed, Martin and Happer1997), interRAI assessments and Clinical Assessment Protocols (Morris et al., Reference Morris, Berg, Bjorkgren, Finne-Soveri, Fries and Frijters2010), the Scott Fall Risk Tool, and the John Hopkins Fall Risk Tool (Poe & Dawson, Reference Poe and Dawson2018). These tools, among many others, have been subject to validity and reliability studies, with varying results. This study will focus on comparing two of these validated tools, as well as an internally developed fall risk tool used with the study’s population.
Method
Study Design, Setting, and Data Sources
A retrospective cohort design study using secondary data was undertaken to compare the accuracy of the interRAI Fall Risk CAP, the Scott Fall Risk Tool, and a supplementary FRA, in predicting future falls. Data sets included the interRAI MDS 2.0 assessments, the supplementary FRAs, and fall incident reports for residents from 18 LTC homes in Nova Scotia and New Brunswick. These data were extracted from medical records housed in the participating homes’ electronic health record system. Records were de-identified, so participant recruitment and consent were not required.
Study Participants
Using secondary data sources, a sample of 1,921 residents across 18 LTC homes in Nova Scotia and New Brunswick, who had interRAI assessments completed between March 1, 2015 and September 29, 2016, were considered for this study (Figure 1). Inclusion required that a resident have at least two records of the interRAI and supplementary FRAs completed, and that each pairing was done within 28 days of the other. A 28-day threshold was selected to exclude any updates to the interRAI assessment that may have been completed outside of a normal nursing quarterly review as a result of a significant event (i.e., a major or injurious fall). These assessments and potential CAP triggers are not accompanied by a supplementary FRA, which are only completed during the nursing quarterly review. Residents who did not have both assessments completed within 28 days of each other were excluded (n = 69). Residents who were discharged within 90 days after their most recent assessments were excluded from the study, as there was not an adequate observation period for a fall incident (n = 299). Residents were not excluded based on age or ambulation method. A final cohort of 1,553 residents was included in the study.
Figure 1: Flow of residents through the study
In order to assess the validity of the SFRS, data from a sub-set of these residents were cross-walked to the Scott algorithm to calculate point values. This sub-set of 587 residents from the initial 1,553 was included based on data availability and reliability for each of the risk factors.
Data Collection
InterRAI assessments and the supplementary FRA data were captured electronically for each resident through electronic charting, and were updated quarterly. Both assessments were paired with residents as long as each assessment was completed within a 28-day period of the other. A sub-set of 587 residents from this group was then paired with cross-walked data containing their scores for the SFRS.
For the majority of analyses, each resident’s most recent assessment was used, to capture a variety of new admission as well as longer-stay residents. Fall outcomes data were obtained through incident reporting systems, which record dates and details of fall incidents by resident. Risk levels associated with each of the assessment types were assigned categories of low, moderate, or high, and outcomes were assigned a value of positive outcome = 0 (no fall occurred in the next 90 days) or adverse outcome = 1 (at least 1 fall occurred in the next 90 days). Disease diagnoses were also collected for each record.
Screening tool 1: interRAI Fall Risk CAP
At the time of this study, Nova Scotia and New Brunswick did not mandate a standardized assessment instrument in LTC homes; however, the interRAI LTCF assessment is becoming widely adopted by many service providers, along with its suite of Clinical Assessment Protocols (interRAI CAPs) (Hirdes et al., Reference Hirdes, Mitchell, Maxwell and White2011). The interRAI CAP that assesses risk of falling in LTC relies solely on a previous history of falls and categorizes residents into low/none, moderate, and high-risk categories. This is based on an assumption that other risk factors for those without a history of falls will be addressed through other CAPs that would be triggered through other items in the assessment instrument (e.g., there are two continence-related CAPs). The low/no fall risk category is used if the resident has had no record of falls in the previous 180 days, the moderate is used if one fall has been recorded, and the high-risk category is used if multiple falls have been recorded.
This history of falls is captured through a regular reassessment (typically quarterly) of each resident based on the complete LTCF instrument. CAPs, such as the Fall Risk CAP, are “triggered” based on responses to the items on the instrument. The CAPs then trigger a much more detailed review of the clinical issue, with guidelines for clinicians to use in their review of any other factors that should be considered in the development of the plan of care.
With the intent of augmenting the interRAI instrument and Fall Risk CAP, some LTC homes use a variety of supplementary FRA tools to capture additional risk factors. Many of these tools consider a wider number of variables such as age, acute and chronic illnesses, and fear of falling (Hnizdo, Archuleta, Taylor, & Kim, Reference Hnizdo, Archuleta, Taylor and Kim2013). They are also completed through regular, ongoing assessments and typically done in tandem with the LTCF assessment instrument. Both tools are then used in combination to develop care plan goals. Other methods for assessing risk for falls are emerging such as video capture, with promising results (Viera, O'Rourke, Marck, & Hunter, Reference Viera, O'Rourke, Marck and Hunter2013; Woolrych et al., Reference Woolrych, Zecevic, Sixsmith, Sims-Gould, Fedlman and Chaudhury2014). However, the additional predictive utility and psychometric properties of these supplementary efforts are rarely considered.
Screening tool 2: SFRS
The SFRS was developed in 2012 and includes 11 risk factors, one of which is protective against falls (resident is immobile) (Scott, Reference Scott2012). A complete list of the risk factors in this screen is included in Table 1. The risk factors are assigned weighted point values based on a review of the literature. For example, a history of previous falls is generally agreed to be the most common and predictive risk factor for predicting future falls, so it is worth 6 points whereas generalized weakness is worth 1 point.
The 10 harmful risk factors combined provide a risk score out of a possible 19 points and the protective factor of being immobile can reduce that value by 5 points. Generally, a score of less than 7 would indicate a normal level of risk for falling in an LTC setting, and universal fall prevention strategies should be in place. A score of 7 or greater indicates a high level of risk and a score of greater than 12 indicates a very high level of risk and possible unsafe ambulation.
Table 1: Characteristics of non-fallers and fallers
Note. SD = standard deviation.
Screening Tool 3: Modified Fall Risk Tool – Nova Scotia Fall Prevention Program
The supplementary FRA used in this study is a modified FRA tool that was developed as part of a Fall Assessment Framework implemented in Nova Scotia (Nova Scotia Health Authority, 2006). To develop this tool, a literature review was conducted on the various fall assessment tools available, as well as related reliability and validity studies. The conclusion of this review was that no one single tool was supported for use in the various settings the program targeted. Therefore, the program team developed a tool using the risk factors that were most prominent in the research (Nova Scotia Health Authority, 2006).
In the FRA, residents are evaluated on 14 risk factors (Table 2), with each factor carrying an equal weight of 1 point. Residents who score less than 4 points in total are considered to have a normal level of risk related to falls in LTC and should have universal fall risk interventions in place. An assessment totaling 4–10 points signifies a higher level of risk and an assessment greater than 10 points signifies a very high level of risk and potential unsafe ambulation.
Table 2: Odds ratios and prevalence by risk factors included across three fall risk assessment tools
Note. aMove to facility in past month is not applicable, as residents were included in study only if they had at least two of each of the interRAI assessments and the supplementary fall risk tools completed. CAP = clinical assessment protocol.
Data Analysis
Data were analysed using the statistical software R version 3.4.0. Logistic regression was completed to evaluate the general predictive value of the various risk factors in each assessment type, and then odds ratios were calculated for each risk factor. Sensitivity and specificity analysis was conducted to further evaluate the predictive validity of each assessment and the c-statistic was used to describe overall accuracy. Residents were assessed as being at low, medium, or high-risk based on the individual thresholds described in each of the tools. For the purpose of this study, the significance level was set at p < 0.05.
Ethical Considerations
The secondary analysis completed in this study was conducted solely within the participating LTC homes systems; therefore, a formal ethics review was not required. The participating LTC homes received their own internal ethics clearance using internal mechanisms.
Results
Sample Characteristics
A final cohort of 1,553 residents was included in this study from the initial 1,921 considered and 69 residents were excluded because their interRAI FRA and supplementary FRA were not completed within 28 days of each other. An additional 299 residents were excluded because they were either discharged or deceased within 90 days of their most recent assessment. General demographic characteristics of the study participants, including proportions of fall risk factors by risk assessment and level, were assessed for this study and are shown in Table 1. The 1,553 residents had a mean age of 82.1 years (standard deviation [SD] ± 13.9) and 68.7 per cent were female. During the 90-day period after the included assessments, 422 (27.2%) of the residents experienced a fall, for a total of 853 recorded falls. Of the residents who fell, 81 (19.2%) fell three or more times.
Risk Factor Predictive Validity
Table 2 contains odds ratios and prevalence in the study population, for each risk factor included in the interRAI Fall CAP, the supplementary FRA, and the SFRS. Consistent with previous findings, a history of falls was the best predictor of future falls, with an odds ratio of 5.06 (95% confidence interval [CI] of 3.96–6.48); 25.8 per cent of the study participants had this risk factor present. Environmental concerns were another fall risk factor that had a significant odds ratio, at 1.68 (95% CI 1.27–2.32); 11.8 per cent of the participants had that risk factor present. The majority of the fall risk factors, however, did not present a higher risk of falling, and some had a protective effect on risk of falling (e.g., physical restraints, immobility).
Table 3 demonstrates logistic regression results and further validates the significance of the history of falls as a risk factor in having a future fall. It is the only risk factor that has statistical significance (p < 0.05) aside from the protective effect of restraint use on a future fall.
Table 3: Logistic regression on fall risk factors across three fall risk assessment tools
Note. Move to facility in past month is not applicable, as residents were included in study only if they had at least two of each of the interRAI assessments and the supplementary fall risk tool completed. SE = standard error; OR = odds ratio.
Assessment Tool Predictive Validity
Figure 2 visually depicts the fall rates associated with each risk assessment tool. In the interRAI Falls CAP, there was a clear gradient in the proportion of residents who fell as the level of risk increased. Among residents assessed as low or no risk of falling (no previous history of falls), 18.2 per cent experienced a fall in the following 90 days. Among those assessed as high risk, 75.4 per cent fell in the same period. On the other hand, the proportion of residents who experienced a fall after being assessed with the supplementary FRA tool showed less variability with the expected level of risk. Among residents assessed by the FRA as low, moderate, and high risk, 16.1 per cent, 28.3 per cent, and 29.1 per cent fell in the following 90 days, respectively. The SFRS has a similar progression, with 15.0 per cent of low-risk residents experiencing a fall in the following 90 days compared with 22.1 per cent of the moderate risk residents and 33.9 per cent of the high-risk residents.
Figure 2: Proportion of residents who fell within 30 days of assessment, by assessment type and risk level
The supplementary FRA tool and the SFRS generally cast a wider net in terms of identifying residents who are at moderate to high levels of risk for falling. Of the study population, 1,253 or 80.7 per cent were identified as at moderate risk and another 151, or 9.7 per cent of the population, were identified as at high risk using the supplementary FRA tool. Through the FRA tool, 90.4 per cent of the population would require more than the universal fall intervention strategies that are put in place for residential LTC residents. Of the resident population studied, 3.7 per cent were in the high-risk category of the interRAI CAP, 9.7 per cent were in the high-risk category of the supplementary FRA and 19.1 per cent were in the high-risk category in the SFRS.
Of the 587 residents who could generate the SFRS scores, 42.2 per cent were identified as being at moderate or high risk of falls, requiring more than the standard universal fall prevention strategies. Conversely, the interRAI CAP identified 25.8 per cent of the population as being at moderate or high risk for falls, with the assumption that the remainder of the residents would benefit from universal fall prevention strategies. This appears to be more in line with national benchmark results, in which 15.7 per cent of LTC residents had experienced a fall in the past 30 days (Canadian Institute for Health Information, 2016a). It is also consistent with past analyses of interRAI data (interRAI, 2010).
Statistically, the predictive validity of each fall risk assessment tool’s categorization of residents as being at low/no or high risk for falling was evaluated through sensitivity, specificity, and area under the curve (AUC). Receiver operating character (ROC) curves are commonly used to evaluate diagnostic tests (Bradley, Reference Bradley1997), plotting the true positive and false positive rates (Fawcett, Reference Fawcett2006). The AUC ranges from 0 to 1.0, with higher values (> 0.5) indicating more accurate diagnostic tests. The true positive rate reflects the sensitivity rate of a diagnostic test, or its ability to accurately identify those with the disease. Specificity, or the true negative rate, describes the test’s ability to correctly identify those without the disease. Sensitivity and specificity values range from 0 to 1.0. Rates greater than 0.70 in sensitivity or specificity were considered indicative of high predictive validity (Barker et al., Reference Barker, Nitz, Choy and Haines2009; Oliver, Daly, Martin, & Mcmurdo, Reference Oliver, Daly, Martin and Mcmurdo2004) and the AUC represented through the c-statistic provided an evaluation of overall accuracy.
As shown in Table 4, none of the three tools had both high sensitivity and high specificity. The interRAI Fall Risk CAP had higher overall accuracy with a c-statistic of 0.673; however, it is less sensitive at 0.502 than the other tools. The supplementary FRA has very high sensitivity at 0.943, but this comes at the cost of very low specificity and poor overall accuracy of only 0.529. The SFRS had moderate sensitivity and specificity and performed somewhat better than the supplementary FRA, with a c-statistic of 0.609. These results are shown graphically in Figure 3.
Figure 3: Receiver operating characteristics curves. Curves compare the three fall risk assessment tools
Table 4: Specificity and sensitivity by fall risk assessment (FRA) type
Note. C-statistic X disease controls for wheelchair use and includes diagnoses of multiple sclerosis, Parkinson’s disease, Alzheimer’s disease, cardiovascular disease, and chronic obstructive pulmonary disease. CAP = clinical assessment protocol.
Multivariate analysis that added various disease diagnoses to the models for all three risk assessments were also evaluated and referred to as “c-statistic × disease” in Table 4. A model that included the interRAI CAP, as well as diagnoses of Parkinson disease, multiple sclerosis, Alzheimer’s disease, chronic obstructive pulmonary disease (COPD), and cardiovascular disease, and controlling for wheelchair usage, yielded the highest c-statistic of 0.749. A model using these same variables yielded a c-statistic of 0.635, with the supplementary fall risk tool, and a c-statistic of 0.684 with the SFRS. These diagnoses have been previously identified as being strongly associated with fall prevalence in LTC settings (Bansal, Reference Bansal2013; Hirdes, n.d.).
Discussion
The objective of this study was to evaluate the predictive accuracy of three separate FRA tools; the interRAI Falls CAP, derived from the LTCF or MDS 2.0 assessments; the SFRS; and a modified fall risk tool that was implemented as part of a provincial fall reduction strategy in Nova Scotia. Based on the overall evidence of predictive validity, the interRAI CAP appears to be the most appropriate risk assessment tool in terms of accurately predicting fall outcome rates at various levels of risk. High fall risk residents will require more robust interventions, and may present more demands on clinical resourcing to carry out those care plans. The interRAI CAP more accurately identified a smaller proportion of residents needing this level of care, compared with the supplementary FRA or the SFRS. This finding is consistent with previous study results indicating that a history of falling is the best predictor of future falling Poss (& Hirdes, Reference Poss and Hirdes2016).
The proportion of residents categorized as being at low or no risk of falling who actually fell within the following 90 days was relatively consistent across the three assessment tools. Although the fall rates were comparable, the percentage of residents who were categorized as low or no risk differed greatly at 74.2 per cent, 9.6 per cent, and 57.8 per cent for interRAI CAP, the supplementary fall risk tool, and the SFRS, respectively. This indicates that the interRAI does a better job at identifying residents who have normal levels of fall risk given their health status and the care setting in which they are placed. Universal fall prevention strategies are appropriate for this group, and some portion of them will still experience a fall. It also suggests that the FRA and SFRS increase the assessment and care planning burden for a population with a low risk of falls. Those resources might be better allocated to the population that actually experiences elevated rates of falls. In other words, the increased sensitivity of FRA and the SFRS reduced rather than improved the predictive validity of fall risk screening compared with the interRAI assessment.
Incorporating certain disease diagnoses may increase the predictive accuracy of the interRAI CAP to further to capture risks associated with LTC residents who have no known history of falls. Of this study’s sample, there were 211 residents who did not trigger the interRAI CAP but did have a fall. It may be useful to examine options to differentiate the non-fallers in the group that did not trigger, because 172 of those residents had at least one of the disease diagnoses associated with future falls. Consistent with previous studies, these results indicate that all three screening tools have limited accuracy in predicting falls, if a history of falling is not present (Latt et al., Reference Latt, Loh, Ge and Hepworth2016); Poss & Hirdes, Reference Poss and Hirdes2016).
These results may also indicate that the distinction, in both research and practice, of screening and assessing, is a valuable exercise. The interRAI CAP’s ability to simply and quickly identify low, moderate, or high levels of fall risk based on very few factors may make it a very appropriate screening tool. Additional factors that are currently on the supplementary FRA and the SFRS did not result in better prediction of future falls. Most of these risk factors are included in the interRAI LTCF assessment instrument, and they are addressed by accompanying CAPs dealing with issues such as continence, problematic medication use, restraints, cognition, and functional status. In practice, clinicians who identify a resident as being at risk should be reviewing other factors in the overall assessment with interdisciplinary teams, in order to develop an effective care plan.
From a clinical point of view, the additional fall risk assessment tools did not augment the interRAI CAP’s value in predicting future falls. Rather, they increased the false positive rate for future falls. From an economic point of view, the additional staff time to complete these supplementary fall risk appraisals, to plan care with reduced accuracy, and to allocate resources to a group with de facto low risk levels yields increased costs to the organization with potentially little to no return on investment. A more appropriate approach would be to use the interRAI Falls CAP triggers to identify risk groups and then to employ the CAP guidelines and routine clinical judgement for a comprehensive risk management approach for these more carefully targeted individuals.
Limitations
This study did not consider the interventions that were put in place to prevent future falls. Perhaps there was some difference in the strategies used by the type of FRA that was done, but that is unlikely to explain the present results, because all three strategies employed history of falls.
An additional limitation of this study is that the cross-walk comparison with the SFRS was imperfect to a degree. The tool does incorporate clinical judgement in one of the risk factors, and because it was not actually used in these facilities, that aspect would not be accurately reflected in the analyses. For example, the risk factor of “more than two falls in previous 6 months” (which was captured in the analyses), also adds “… and/or clinical judgment of high risk” (which was not captured). This was considered to be a minor limitation, as that clinical assessment of “high risk” would have likely been based on at least one of the other risk factors. In addition, that might have increased the sensitivity of this cross-walk further, but not necessarily the specificity, which was problematic.
The use of secondary data that were not collected to answer the specific research question has some potential to introduce bias into this study, including misclassification. This risk is assessed as low, as risk levels across each of the tools are calculated automatically within the electronic health record system, based on clinician responses to assessment questions. There is some potential, however, that clinicians may not fully complete fall risk assessments, if they believe a resident is truly not at risk based on their clinical judgement. Routine clinical judgement will, and often should, introduce variation from algorithmic approaches to risk assessment.
No measures were required in this study to address missing data, as assessment fields are mandatory with the electronic health record system where these data were captured. Data quality may still be an issue if clinicians do not accurately complete assessments. The interRAI assessments have undergone significant repeatability and reliability studies (Hongsoo, Reference Hongsoo2015), indicating that this risk is low.
Although attempts were made to control for potential confounding disease diagnoses such as Parkinson’s disease or multiple sclerosis, there are potentially unmeasured confounding variables that could limit the internal validity of this study. The researchers did not have access to variables such as environmental factors not specific to the resident such as room privacy type, location, and type of furniture, equipment, or flooring in rooms or common areas. Other potential confounders that were unmeasured include levels of physical activity, staffing levels during fall incidents, and types of footwear used by the resident.
Further Studies
It may be useful in future research to examine if different intervention strategies are employed at the various risk levels, across the three fall risk screens. Although each screen tool incorporates a history of falls, additional risk factors present in the SFRS and the supplementary FRA may prompt alternative interventions. Additionally, because the cross-walk comparison with the SFRS was somewhat imperfect, a study population in which the full tool was actually used in parallel to the interRAI assessment could provide a more accurate assessment of its validity.
Another potential study area would be to consider other interRAI CAP areas that are triggered in this population in relation to the fall CAP and fall outcomes. The interRAI Fall Risk CAP is narrow in focus by design, as many other common fall risk factors such as medication interaction, strength, and balance, are intended to be addressed through various other CAP areas such as physical activities promotion, activities of daily living, and appropriate medications. Further research, including additional studies of disease diagnoses in predicting falls, may indicate how effectively those various CAP areas are being addressed to reduce overall risk of injury from falls. There may also be value in attempting to further differentiate the low-risk group to identify those with none versus some of these additional risk factors.
Conclusion
None of the FRA tools studied performed at a high level for both specificity and sensitivity. However, no risk factors included in the supplementary FRA tool or the SFRS provided greater predictive ability than the factor of falls history that is addressed in the interRAI CAP. These findings indicate that the supplementary fall risk tool and the SFRS may not provide additional clinical value in predicting falls, and the economic considerations associated with over-triggering high cost fall interventions could be considerable. These results indicate that a change in practice may be appropriate, whereby the interRAI Falls CAP is used to maximize true positives, and clinical judgement augments that tool to minimize false negatives. Predicting falls among seniors, especially those with an unclear or absent fall history, is complex. A more balanced approach of doing comprehensive FRA and clinical management in carefully targeted sub-populations may be more appropriate for LTC residents.
