Finding the balance between lumping and splitting. A response to Martin et al.

We appreciate the careful examination of sleep–wake symptoms in our paper (Forbes et al., 2023) by Martin, Lopez, Micoulaud-Franchi, and Gauld (2024). The authors highlight a number of important points of difference in which symptoms were ‘lumped’ v. ‘split’ in our output compared to the results in Gauld et al. (2021a) – an earlier study that aimed to inventory all of the different symptoms of the sleep–wake disorders. Martin et al. (2024) also highlight that some of these differences have important clinical implications for differential diagnosis and treatment planning for sleep–wake disorders. Overall, it seems likely that the different aims of our studies (mapping the patterns of repetition throughout the DSM v. creating an inventory of symptoms from a clinical sleep medicine perspective) led us to err on different ends of the lumping v. splitting spectrum.

We agree with Martin et al. (2024) that the results of our analysis will not be appropriate when differential diagnosis or treatment plans are a focus – in particular because we removed key diagnostic information about symptom onset, duration, frequency, and severity, as well as causes and consequences including associated distress and impairment. We also agree that putting more weight on specialized clinical expertise in each chapter of the DSM would likely have generated different results and added important nuance, thereby emphasizing smaller differences between symptoms embedded in the context of their diagnosis (i.e. erring on the side of splitting). By contrast, our approach likely overemphasizes similarities among diagnoses and chapters by removing symptoms from their context (i.e. erring on the side of lumping). For example, insomnia due to withdrawal from a substance is meaningfully different from insomnia due to worrying, but they are treated as the same in our analysis. Indeed, the natural language processing step in our analyses was solely used as a safety net to check for any matches we might have missed, so throughout our methods we erred on the side of lumping rather than splitting symptoms.

Given our aim was to map symptom repetition in the DSM-5, it is interesting to consider the differences between Forbes et al. (2023) and Gauld et al. (2021a), as highlighted by Martin et al. (2024), from the perspective of this aim. Some of the differences in lumping v. splitting would make little to no difference to the patterns of symptom repetition because they relate to symptoms that do not repeat at all, such as cataplexy and abrupt terror arousals. Several would have resulted in minor differences within the sleep–wake chapter, such as in one additional symptom repeating in the sleep–wake disorders (i.e. altered oneiric activity and behavioral symptoms during the night) or two fewer (non-repeating) symptoms being listed in restless legs syndrome. The extraction of daytime fatigue in sleep apnea as separate from daytime sleepiness would have resulted in one additional cross-chapter symptom repetition, bringing the total number of occurrences of fatigue to 11, spanning seven chapters (compared to 10 occurrences spanning six chapters in Forbes et al., 2023).

A noteworthy exception to these relatively minor differences is insomnia, which Forbes et al. (2023) listed as the most commonly repeating symptom throughout DSM-5 (Table 2 of Forbes et al., 2023; occurring in 22 diagnoses spanning eight chapters). Martin et al. (2024) highlighted that, in contrast to Gauld et al. (2021a), we lumped three classically distinct insomnia symptoms – ‘difficulty initiating sleep’, ‘difficulty maintaining sleep’, and ‘waking up earlier than desired’. In a parallel project where we were measuring these symptoms in a self-report survey (see Forbes et al., in press), we had both the split and lumped versions of these insomnia symptoms assessed during the piloting stage of the survey, and choosing the lumped version of the symptoms was one of the last changes we made to the item pool. However, examining the pilot data from several hundred undergraduate students corroborates Martin et al.'s (2024) point that these symptoms are meaningfully different – in line with their description in the International Classification of Sleep Disorders as distinct diagnostic criteria (Gauld et al., 2021b) – with correlations among the split items ranging from r = 0.35 to 0.65, and correlations for the split items with the lumped item ranging from r = 0.45 to 0.77 (see Table 1).

Table 1. Pearson correlations among the pilot items that split the three insomnia symptoms v. the lumped symptom

Note. All ps < 0.001.

Martin et al. (2024) also pointed out that we did not extract mentions of ‘awakening’ in diagnostic criteria as indexing insomnia. For example, in nightmare disorder we did not code the diagnostic criterion ‘On awakening from the dysphoric dreams, the individual rapidly becomes oriented and alert’ as including insomnia as a symptom.

If we reconsider our results incorporating both the tripartite symptom split for insomnia, and the three instances of ‘awakening’ noted by Martin et al. (2024), the symptoms would occur with the following frequency: insomnia in 16 diagnoses (six chapters), difficulty initiating sleep in four diagnoses (three chapters), difficulty maintaining sleep in seven diagnoses (three chapters), and waking up earlier than desired in three diagnoses (three chapters). As a consequence, insomnia would be ranked fourth in Table 2 of Forbes et al. (2023), rather than first, and the difficulties maintaining sleep in rapid eye movement sleep behavior disorder, nightmare disorder, and restless legs syndrome would no longer be omitted.

While there is clear merit to this approach, we do think that the umbrella ‘insomnia’ symptom was the right aggregation level for our aim of mapping symptom repetition because it is the most common description of sleep disturbances throughout the DSM (i.e. ‘insomnia’ is listed in the diagnostic criteria for 16 diagnoses spanning six different chapters). Although generalized anxiety disorder, post-traumatic stress disorder, and acute stress disorder specify ‘difficulty falling or staying asleep’ and the melancholic specifier for depressive and bipolar or related disorders specify ‘early-morning awakening with inability to return to sleep’, the only other instances of the sub-symptoms occur in the sleep–wake disorders chapter.

Overall, because our aim was to map the patterns of overlap among the symptom criteria, we believe that our approach struck a reasonable balance – perhaps erring on the side of over-lumping. We would be delighted to see more work like that of Martin et al. (2024) applying deep clinical expertise to arrive at different conclusions, and have made our data publicly available (https://osf.io/w3kcn) with this kind of work in mind. Other directions to consider could include exploring the features of the DSM diagnostic criteria we removed – like frequency, duration, episodicity, and known causes (e.g. trauma or substance use) – to develop a more nuanced conceptualization of symptom repetition that could have utility beyond description, perhaps even for use cases as complex as differential diagnosis and treatment planning.

Funding statement

This work was supported by an Investigator Grant from the Australian National Health and Medical Research Council (grant number GNT1194292).