Preface
It was well beyond my expectations when I heard from Professor Ames that our meta-analysis paper (Han et al., Reference Han, Cole, Bellavance, McCusker and Primeau2000) was among the top-cited papers in International Psychogeriatrics. As an expression of my gratitude to the editor for the invitation and to the audience of this paper, I would like to offer this informal discussion on the background of the paper and extend a few ideas that were not conveyed fully at the time, due either to insufficient knowledge or unavailability of relevant data or methodologies.
How did the meta-analysis paper come to be born?
As far as I can recall, the idea for writing a meta-analysis paper on cognitive decline was inspired by my mentor, Dr. Martin Cole, during my geriatric research fellowship. Dr. Cole is a renowned Canadian geriatric psychiatrist and researcher as well as an advocate of evidence-based-medicine. Following a staff journal club on Ritchie's article, “Establishing the limits of normal cerebral ageing and senile dementias” (Ritchie, Reference Ritchie1998), I realized that although the Mini-mental State Examination (MMSE) had been used routinely in many geriatric and dementia clinics around the world, its usage was usually limited to cross-sectional assessments, serving as a “present status” examination for screening for cognitive impairment or dementia. The value of serial MMSE scores seemed to be less well appreciated in the clinical decision-making process, even though such data are often available for the same patients. For example, a MMSE score of less than 24 has been adopted as a conventional cut-point for mild dementia, but no comparable quantitative threshold had been established for “abnormal” or “accelerated decline” using the MMSE. Consequently, when counseling patients and their families, clinicians often faced a great deal of uncertainty as to whether or not a reduction of, say, 2 MMSE points from the previous year would indicate a significant worsening of function or a normal fluctuation. Did this absence of a quantitative threshold for cognitive decline reflect a lack of longitudinal studies that estimated the rate of cognitive decline, a lack of consensus among researchers regarding the meaningfulness and interpretability of a small decline, or the reluctance of clinicians to adopt such a quantitative index in clinical practice, due to the harsh criticism of the MMSE for its insensitivity to small changes in cognition? Dr. Cole suggested it was the right time to conduct an evidence-based review of the literature on this topic, so that clinicians and researchers could benefit from a pooled estimate for typical rate of cognitive decline for Alzheimer's disease (AD) patients and from what we might learn about potential risk factors for an accelerated or decelerated cognitive decline.
Another motivation arose from a practical concern, akin to a sense of urgency. In the 1990s, anti-dementia drugs, cholinesterase inhibitors (ChEIs), had appeared on the market (Hogan and Patterson, Reference Hogan and Patterson2002; Herrmann, Reference Herrmann2002). If these promising anti-dementia treatments came to be widely used, we may no longer be able to determine the typical rate of cognitive decline for AD, free of “publication bias”, because both the control groups of randomized clinical trials and observational cohorts might be “contaminated” by treatment effects, and hence, manifest a slower rate of cognitive decline than expected during the natural course of AD.
What are the most cited findings from the meta-analysis paper?
To answer this question, I did a mini “meta-analysis” on the 33 citations (by June 2008) of the paper. The citations came from a wide range of peer-reviewed journals, including geriatric/gerontological journals (n = 18), other specialty or general clinical journals (n = 10) and other biomedical or life science journals (n = 5; e.g. Journal of Neural Transmission, Supplementum). Excluding eight review or editorial articles, there are 25 original studies that followed participants longitudinally with the MMSE at two or more occasions. Of these, 12 involved evaluation of the efficacy or effectiveness of ChEI or other drug (e.g. memantine) treatments using data from randomized clinical trials or observational cohorts. These studies cited our pooled estimate of 3.3 MMSE points for annual rate of change (ARC) as a reference or historical control value when making inferences about the effect of ChEI treatment. For instance, Wallin and colleagues (Reference Wallin2007) followed 435 outpatients diagnosed with AD who received donepezil for three years in routine geriatric clinics in Sweden and observed a total decline of 3.8 points (95% confidence interval: 3.0–4.7) or 1.3 points per year. Due to the absence of a control group, these authors used our estimated ARC as an historical reference for untreated patients and concluded that donepezil was effective in reducing cognitive decline. Similarly, Grossberg and colleagues (Reference Grossberg, Irwin, Satlin, Mesenbrink and Spiegel2004) followed 2010 enrollees with probable AD in two open-label extension studies of randomized clinical trials of rivastigmine for up to two years. Again, they concluded that rivastigamine-treated patients performed significantly better at one year (mean MMSE decline: 1.3 versus 3.3 to 3.6 points) and two years (mean MMSE decline: 4.3 versus 5.3 to 6.6 points) than patients in several historical control groups, including the pooled ARC estimate from our meta-analysis.
Therefore, it seems that a well-received point from our meta-analysis was the pooled ARC estimate. It filled a gap between clinical practice and drug trials, and provided an acceptable reference or “historical control” for evaluating drug efficacy when a no-treatment control group was either no longer available or ethically unacceptable. Interestingly, patients in certain Canadian provinces are reimbursed for ChEI treatment from the publicly funded drug benefit plans only if the course of the illness is monitored with the MMSE (Hogan and Patterson, Reference Hogan and Patterson2002; Hermann, 2002). In these plans, the “pooled” ARC estimate from the meta-analysis seems to be conceptually appealing as the “best available evidence” for documenting the disease progression, evaluating treatment efficacy and justifying patient need for continuing therapy. In fact, based on the pooled ARC estimate, it is recommended that no change or an improvement over one year on the MMSE be viewed as evidence of benefit (Hogan and Patterson, Reference Hogan and Patterson2002).
On the other hand, it is unclear whether the pooled ARC estimate has been adopted by clinicians as a desktop reference for assessing patient disease progression and effectiveness of treatment. Admittedly, use of the ARC or change score in individual patients is more challenging than for population comparisons. In addition to greater between-patient variation, calculating a change score per unit of time requires that the clinicians review and synchronize serial test scores on each patient, which may be an onerous burden unless the repeated MMSE assessments are incorporated into a computerized clinical data collection system. Apparently, this is an untouched area for future investigation.
What might we have done differently?
It is always interesting to look back on what was done ten years ago. At the time, we regretted that our meta-analysis did not detect any significant factors that may have accelerated or decelerated the rate of cognitive decline of AD. We suspected that this “failure” was attributable to the variation of the ARC estimates across studies. Now, after nearly a decade, we reflect that at least for some of the demographic risk factors for which the sample sizes were adequate (e.g. gender and education), the absence of prediction may have correctly revealed rather than hidden the truth. As demonstrated in several more recent studies, gender (Mendiondo et al., Reference Mendiondo, Ashford, Kryscio and Schmitt2000; Backman et al., Reference Backman2003; Suh et al., Reference Suh, Ju, Yeon and Shah2004) and education (Backman et al., Reference Backman2003; Suh et al., Reference Suh, Ju, Yeon and Shah2004) are not independently associated with the rate of cognitive decline in AD patients, though cross-sectionally both factors may affect a person's MMSE performance and deserve specific consideration when establishing the population norm. One possible explanation for this conflicting finding is that such fixed risk factors mainly affect the intercept or start point rather than the slope of the cognitive deterioration in AD (Crystal et al., Reference Crystal1996; Backman et al., Reference Backman2003). Because we adjusted for baseline MMSE score in the random effect regression model, the residual effect of these fixed risk factors on cognitive decline would be removed. If confirmed, this view would reinforce our initial recommendation that the MMSE score at the diagnosis (or at study entry) be considered when comparing the rates of cognitive decline on the MMSE between different patients or across different populations, or when searching for risk factors of cognitive decline in observational studies. Apart from enhancing comparability, adjustment for baseline MMSE level may help control cumulative confounding of unknown factors or events that might have operated on the patients’ cognition since the underlying neurodegenerative process started, and hence, partially remedy our inability to trace the true “time zero” of the disease process. Another practical implication of this recommendation is that after adjusting for baseline cognition in a regression model, gender and education may be left out of the model to save statistical power for evaluating more biologically plausible or disease-specific risk factors, such as Apolipoprotein E genotypes, age at onset and cardiovascular comorbidities etc.
Another regret is that we focused on only the first and last MMSE assessments from each patient and did not address potential non-linear trends of ARC over time. At the time, most studies did not report data for more than two assessments and the assessment intervals varied greatly across studies. Given that a non-linear trajectory of decline in AD is a finding in the literature (Mendiondo et al., Reference Mendiondo, Ashford, Kryscio and Schmitt2000; Grossberg et al., Reference Grossberg, Irwin, Satlin, Mesenbrink and Spiegel2004; Wallin et al., Reference Wallin2007) and that, in our meta-analysis, the accuracy of the ARC estimate was affected by a fewer number of assessments and a heterogeneous baseline MMSE, this is an area deserving further investigation. Fortunately, the proliferation of longitudinal studies using repeated measures of the MMSE or other instruments in the past decade and the rapid progress in statistical methodologies for modeling the natural course of the disease (Mendiondo et al., Reference Mendiondo, Ashford, Kryscio and Schmitt2000; Teipel et al., Reference Teipel, Mitchell, Moller and Hampel2007) makes this goal more achievable than ever. Despite the possibility of using these advanced methodologies to obtain more accurate estimates of ARC for different stages of the disease, caution should be exercised in extrapolating our pooled ARC estimate to AD patients at a very late or very early stage of the disease or with very different demographic and clinical profiles. This is consistent with the recommendation for reducing patient heterogeneity to enhance the validity and precision of the MMSE assessments (Folstein, Reference Folstein2007). In fact, a careful consideration of patients’ heterogeneity in terms of major disease markers, such as staging, duration and treatment, or more insightful interpretation of its clinical meaning beyond the psychometric limitations of the MMSE. For instance, a lack of “practice effect” beyond three months may itself be indicative of a pathological deterioration process often seen in dementia patients (Helkala et al., Reference Helkala2002), whereas a lack of expected decline over an extended follow-up interval of one to three years in established dementia patients may result from an unrealized protective factor, such as ChEI use, rather than solely the inherent insensitivity of the MMSE (McCarten et al., Reference McCarten, Rottunda and Kuskowski2004).
Concluding remarks
In a synopsis published in a 1998 issue of the International Journal of Geriatric Psychiatry, Brayne (Reference Brayne1998) discussed the reasons for the popularity of the MMSE and anticipated another decade of heavy quotations. Are we going to see yet another decade of high citation rates of the MMSE? The answer seems obvious. Both clinicians and researchers are still committed to a rate of cognitive decline as measured by the MMSE in understanding AD (Mendiondo et al., Reference Mendiondo, Ashford, Kryscio and Schmitt2000; Soto et al., Reference Soto, Gillette-Guyonnet and Vellas2005; Folstein, Reference Folstein2007). This rate of cognitive decline appears to be fundamental to establishing the boundary between normal aging and dementia, improving early detection and diagnosis and monitoring of disease progression and treatment efficacy. As modern neuroscience is advancing into the final frontier of the seemingly invincible AD, there will be increasing demand for psychometrically sensitive, yet clinically feasible, new tools to quantify the cognitive “phenotype” of AD (Irizarry et al., Reference Irizarry2008). In the meantime, the MMSE and its ARC estimate, however diminutive and imprecise, will remain useful and affordable measures as well as valuable prototypes.
Acknowledgments
I would like to extend my heartfelt gratitude to Dr. Martin Cole and Dr. Heather Allore for their critical reviews and invaluable comments on an early version of this paper.
A quick glance through an issue of International Psychogeriatrics pulled from my shelves at random (20 (5), October 2008), revealed that of 16 review or original research articles, five (31%) (Lyketsos et al., Reference Lyketsos, Szekely, Mielke, Rosenberg and Zandi2008; Mendes-Chiloff et al., Reference Mendes-Chiloff, Ramos-Cerqueira, Pereira Lima and Rodrigues Torres2008; Milne et al., Reference Milne, Culverwell, Guss, Tuppen and Whelton2008; Nakaaki et al., Reference Nakaaki2008; Orrell et al., Reference Orrell, Hancock, Galboda Liyanage, Woods, Challis and Hoe2008) cited Folstein's MMSE. It therefore caused me little surprise to find that that the valuable contribution of Han et al. (Reference Han, Cole, Bellavance, McCusker and Primeau2000) in establishing a typical rate of decline for AD patients assessed with the MMSE had received the equal sixth highest number of citations of any paper published in the journal up to the end of 2006. Its importance is underlined by the arbitrary use of MMSE cut-points for regulating the starting and cessation of cholinesterase inhibitors according to the National Institute of Health and Clinical Excellence (NICE) guidelines in the U.K. and the Pharmaceutical Benefits Scheme (PBS) criteria for reimbursing cholinesterase inhibitors in Australia, to give but two examples (Ritchie et al., Reference Ritchie, Ames, Masters and Cummings2007; Ames et al., Reference Ames, Kaduszkiewicz, van den Bussche, Zimmerman, Birks and Ashby2008). Marshall Folstein has told me (and others!) that one major reason for the development of the MMSE was so that the question “how is this patient today?” could be answered by objective measurement of cognition. What Dr. Han and colleagues have done is to allow us to answer the question “how is this patient today compared to how he/she should be?” and that is a major service to the field on which they are to be congratulated. Their work is likely to stand the test of time, as the current widespread use of cholinesterase inhibitors in the management of AD (Ames et al., Reference Ames, Kaduszkiewicz, van den Bussche, Zimmerman, Birks and Ashby2008) means that naturalistic studies of the type analyzed by Han et al. (Reference Han, Cole, Bellavance, McCusker and Primeau2000) will no longer be done on large numbers of untreated patients in the future.
