The Feasibility, Acceptability, and Efficacy of Positive Search Training for Irritable Youth: A Single-Case Experimental Design

Abstract

Although irritability is common in youth, research on treatment is in its infancy. Threat biases are more pronounced in irritable compared to low irritable youth, similar to evidence found in anxious youth. Therefore, interventions targeting these biases may be promising for reducing irritability. This study utilised a multiple baseline case series design to determine the feasibility, acceptability, and efficacy of positive search training (PST) for irritable children. Three children were included who met criteria for a principal diagnosis of Disruptive Mood Dysregulation Disorder (DMDD), and a secondary diagnosis of Attention Deficit Hyperactivity Disorder (ADHD) or Major Depressive Disorder (MDD). PST was feasible with two of the three participants; one child refused to continue after one session. For the two participants who completed PST, acceptability was stable with moderate-to-high ratings of engagement and enjoyment, and high and stable treatment-relevant verbalisations of the key strategies. Both cases showed declines in DMDD severity across treatment and no longer met criteria at post-treatment. Both participants met criteria for Oppositional Defiant Disorder (ODD) at post-treatment (considered less severe for irritability than DMDD). Declines in parent-reported irritability occurred for both cases, however some returns to baseline were observed. Overall, PST for irritable youth shows promise as an acceptable and feasible intervention. Further studies are needed combining PST with strategies for secondary diagnoses, given its high comorbidity with disruptive behaviour disorders.

Introduction

Paediatric irritability in both clinical and community populations has been associated with impairments in social, academic, and psychological functioning (Copeland, Shanahan, Egger, Angold, & Costello, 2014). Childhood irritability is the primary symptom of Disruptive Mood Dysregulation Disorder (DMDD), introduced into the DSM-5 to account for severe and impairing levels of irritability. Irritability also presents across the internalising and externalising spectrum, being both a symptom of, and highly comorbid with many disorders, including Oppositional Defiant Disorder (ODD), Attention Deficit Hyperactivity Disorder (ADHD), Post-Traumatic Stress Disorder (PTSD), anxiety disorders, depressive disorders, bipolar disorder, and Autism Spectrum Disorder (ASD) (American Psychological Association, 2000). The long-term issues associated with high childhood irritability suggest a needed introduction of an intervention that specifically targets the mechanisms underlying irritability in order to reduce the risk of future psychopathology (Stringaris & Goodman, 2009a). Treatment approaches for irritable youth can arguably be refined from interventions targeting related phenotypes of irritability (i.e., anger) and from disorders that share underlying mechanisms (Roy, Lopes, & Klein, 2014; Stringaris, Vidal-Ribas, Brotman, & Leibenluft, 2017), such as biases in attention as observed in anxiety disorders (Waters, Bradley, & Mogg, 2014; Waters, Mogg, Bradley, & Pine, 2008).

Diagnostic Comorbidity

Given the transdiagnostic nature of irritability, its clinical specificity can pose challenges when diagnosing youth. As such, research has demonstrated very high rates of comorbidity between DMDD and ODD (Freeman, Youngstrom, Youngstrom, & Findling, 2016), with irritable mood considered a distinct subtype of ODD (i.e., touchy or easily annoyed, angry) alongside defiance (i.e., argumentative, non-compliant behaviour) (Evans et al., 2017; Stringaris & Goodman, 2009a, 2009b). As prior research has indicated that typically more than 50% of youth in clinical samples of DMDD also meet criteria for ODD (Althoff et al., 2016; Evans et al., 2017; Mayes et al., 2015), the DSM-5 specifies that the severity, frequency, and intensity of irritable mood within DMDD is more severe and impairing than in ODD. Thus, where both clinical criteria are met, only DMDD should be diagnosed (American Psychological Association, 2000). Within samples of youth with ADHD, approximately 20–40% of youth have been found to display elevated or clinical levels of DMDD (Benarous et al., 2017; Eyre et al., 2017; Mulraney et al., 2016; Waxmonsky et al., 2017). Alongside the high rates of comorbidity seen within externalising disorders, youth with anxiety disorders also demonstrate severe irritability (Cornacchio, Crum, Coxe, Pincus, & Comer, 2016; Stoddard et al., 2014). Despite the lack of diagnostic clarity of DMDD, some models of youth irritability (e.g., Brotman, Kircanski, Stringaris, Pine, & Leibenluft, 2017) propose underlying mechanisms that may elucidate key targets for clinical intervention that are not solely explained by its high rates of comorbidity with other disorders.

Underlying Mechanisms

Brotman et al. (2017) proposed that alongside aberrant reward processing, aberrant processing of threat stimuli is a key mechanism that underlies irritability. Much of the research investigating potential underlying mechanisms of irritability in youth has been developed through the application of tasks used in research on cognitive biases in other forms of psychopathology (e.g., anxiety disorders) and phenotypically related populations (e.g., anger). Similar to samples of anxious youth (e.g., Dudeney, Sharpe, & Hunt, 2015; Shi, Sharpe, & Abbott, 2019), biases in attention have been found in both community and clinical samples of youth with heightened irritability (Hommer et al., 2014; Kircanski, White, et al., 2018; Salum et al., 2017). For example, Salum et al. (2017) found that 6- to 12-year-olds with heightened irritability displayed attention biases towards threat stimuli (e.g., angry faces) relative to neutral faces on a visual dot-probe paradigm when compared to non-irritable youth. Research suggests that threat biases are present among irritable youth when covarying for anxiety symptoms (Kircanski, White, et al., 2018; Salum et al., 2017) and that they might maintain irritability symptomology and thus require targeted intervention. Indeed, treatment research within anxious samples has targeted threat attention biases in order to reduce the severity of anxiety symptoms (Pettit et al., 2020; Waters et al., 2015; Waters, Pittaway, Mogg, Bradley, & Pine, 2013). However, no such studies have examined the efficacy of attention bias training approaches within irritable youth.

Intervention Approaches

Interventions specifically targeting irritability in children have been scarce (see Benarous et al., 2017), primarily due to the lack of conceptual models of irritability until recent years (Kircanski et al., 2019). Prior studies have examined pharmacological interventions for irritability with children and adolescents, including the use of stimulants (Blader et al., 2016) and atypical antipsychotics (Krieger et al., 2011). Parent management training has also been employed with parents of children with severe irritability, with parents learning to use consistent positive reinforcement techniques (Comer, Chow, Chan, Cooper-Vince, & Wilson, 2013), similar to approaches in the treatment of disruptive behaviour disorders (Nobel et al., 2020). One novel pilot intervention (Waxmonsky et al., 2013) utilised a combination of stimulant medication, Cognitive Behaviour Therapy (CBT), and parent training for seven children 7–12 years old with ADHD and Severe Mood Dysregulation (characterised by persistently irritable and sad mood). However, treatment approaches have largely failed to directly target mechanisms posited to play a role in irritability. Therefore, applying treatment approaches for disorders that have irritability as an associated feature (i.e., anxiety) and have targeted key underlying mechanisms (i.e., attention biases for threat) show promise for the use within the treatment of irritability (Stringaris et al., 2017).

In a recent review, Kircanski et al. (2019) specified the mechanisms thought to underlie irritability based on the translational model proposed by Brotman et al. (2017) as key treatment targets. They highlighted the utility of testing approaches such as CBT for reducing aberrant information processing biases in irritable youth via exposure to non-reward and/or threat stimuli, alongside parent management training. Tested initially in a feasibility study (see Kircanski, Clayton, Leibenluft, & Brotman, 2018), exposure-based CBT with parent training for youth with DMDD was trialled with 10 youths, with preliminary results suggesting the intervention was feasible, although further testing is required to determine its efficacy (Kircanski, Clayton et al., 2018). One novel intervention that has been introduced to target irritability (Stoddard et al., 2016) and aggression (Penton-Voak et al., 2013) in youth used emotion identification training to alter the point at which youth interpret an ambiguous face as angry or threatening. Individuals were then provided feedback after they identified a face across several trials to increase positive judgements of ambiguous faces, thereby decreasing irritability and aggression. Comparatively, a recent trial of computerised interpretation bias training in a sample of youth with DMDD found no significant difference between the control condition and the active intervention group despite shifts towards labelling ambiguous faces as happy (Haller et al., 2022). Together, these studies support the notion that mechanisms involved in the maintenance of disorders may be considered key targets of intervention research. Alongside emotion identification training, attention control training has been found to be a promising treatment approach for anxiety (Waters et al., 2013, 2015), highlighting the potential for use of these interventions within irritable youth.

Positive Search Training

Following prior studies that have found inconsistencies in the reduction of attention biases and anxiety symptoms when utilising primarily dot-probe versions of attention bias modification (ABM) training, several studies suggest that positive search training (PST) may be an effective form of ABM training (Mogg, Waters, & Bradley, 2017). Specifically, the focus on attending to positively oriented stimuli and inhibiting attention to negative distractors in combination with reward-based engagement strategies may make PST particularly suitable to young children with high levels of irritability.

Waters et al. (2013) examined the efficacy of PST in a sample of 37 anxious children. PST was delivered to participants at home whereby each participant completed a computer-based program in which they were required to search for positive targets within arrays of threatening distractors for four sessions each week across 3 weeks. Visual search arrays consisted of a matrix of angry faces with one happy face presented in each array for a total of 160 training trials. In the control condition, participants were required to search for a bird presented in an array of flowers to ensure that changes were not due to practice. Results indicated that PST led to clinical improvement in anxiety symptoms compared to pre-treatment and compared to those in the control condition.

Similarly, enhanced versions of PST (i.e., Waters et al., 2015, 2016) have involved a visual search task of both positive and calm stimuli among unpleasant distractor images to increase the generalisability of the training across the 12 sessions. Prior to the beginning of training, participants were encouraged to either look for the calm target, look for the good target, or look for both and to verbalise key search strategies in the form of catchphrases expressed as jingles to consolidate and generalise learning and increase their attention to the task. Waters et al. (2015) found reduced parent-reported anxiety and depressive symptoms and reduced clinician-rated diagnostic measures of anxiety in PST compared to waitlist controls. Additionally, youth in the PST group continued to decline in anxiety symptomology post-treatment, with 42% and 75% of youth in the PST condition no longer meeting criteria for their principal anxiety disorder at post-treatment and 6-months post-treatment, respectively. Increased verbalisations of treatment-related strategies (i.e., look for good, look for calm, use both options, and never give up) was associated with greater global functioning at post-treatment, whereas greater verbalisation of non-treatment-related content was associated with lower global functioning at post-treatment. In Waters et al. (2016), clinically anxious children who were allocated to the PST group (n = 22) had significantly greater reductions in anxiety compared to the waitlist control group (n = 19). Within the PST group, 50% no longer met diagnostic criteria for anxiety at post-treatment and 54% no longer met diagnostic criteria at 6-months follow-up. Significant reductions in parent- and child-reported anxiety symptoms, depressive symptoms, and internalising and externalising problems were also found from pre- to post-treatment in the PST group only. Moreover, children in the PST group who verbalised more strategies of treatment-related phrases had fewer diagnoses at post-treatment and greater reductions in the severity of their principal diagnoses at post-treatment and 6-months follow-up.

Following on from early positive outcomes from PST, Waters et al. (2019) compared PST (n = 116), to a classroom-based, therapist-delivered CBT program (n = 127), and a curriculum-as-usual (CAU) control condition (n = 60). Children in PST and CBT had significantly greater reductions in their self-reported anxiety severity compared to the CAU condition from pre- to post-treatment; however, there were no significant differences at 12-months post-treatment. Furthermore, parent-reported anxiety severity declined significantly from pre-treatment to 12-months follow-up in the PST compared to the CAU groups. Finally, research has assessed neural changes with implementation of PST within a sample of 15 clinically anxious youth (Waters et al., 2018). At post-treatment, 77% of the sample no longer met their principal diagnosis of anxiety and 66% no longer met criteria for any diagnosis following PST. Moreover, the severity of the principal diagnosis and global functioning all significantly improved from pre- to post-treatment in the PST group. Significant reductions in neural activation were observed for both angry and happy faces relative to neutral faces from pre- to post-treatment.

Current Study

Building on converging evidence of a mechanistic role of attention control in relation to youth irritability, and the benefits of PST in targeting this mechanism in anxious youth, the present study provides the first test of the feasibility, acceptability, and efficacy of PST for irritable youth. Here, guidelines suggest a focus on pilot testing interventions prior to larger-scale research trials (Campbell et al., 2000; Craig et al., 2008), with single-case experimental designs often used to determine the appropriateness of an intervention prior to wider dissemination (see Chambless & Ollendick, 2001; Gallo, Comer, & Kendall, 2013). One such approach to single-case experimental designs includes the multiple baseline design (Tate et al., 2016), used in prior treatment research (i.e., Smith, Handler, & Nash, 2010) in order to track key symptomology continuously and improve our understanding of the effectiveness of interventions across time. Consequently, this study employed a multiple baseline case series design to examine the fidelity of PST for irritable youth.

Hypotheses

Based on prior research testing the efficacy of PST with anxious youth (Waters et al., 2013, 2015, 2016, 2018), we first hypothesised that administering PST would be feasible within a population of youth with clinical levels of irritability. This was indexed by treatment completion and the ease of delivery (i.e., the technical and practical aspects of PST). Secondly, we hypothesised that PST would be acceptable within this population, indexed by child engagement and enjoyment (based on child and parent ratings of enthusiasm, concentration, enjoyment, and usefulness) and their learning and engagement during the program (based on the number of treatment-relevant strategies children verbalised at the end of the sessions). Finally, we hypothesised that PST would be efficacious for irritable youth. This was indexed by parent and child symptom ratings of irritability, clinician-rated severity of their diagnosis, and improvements in global functioning.

Method

Design

This case-series employed a multiple baseline design across participants with follow-up (Gallo et al., 2013). Participants were randomly assigned to a baseline phase of 2, 3 or 4 weeks. Treatment was initiated following completion of the baseline phase with ongoing monitoring continued from the baseline phase until the completion of treatment. Participants completed follow-up assessments in the post-treatment phase immediately following and at 2-months after treatment.

Participants

There were three participants, including two 7-year-old children (one male, one female) (Case 1 and 2) and one 9-year-old male (Case 3). Based on the DSM-5, Case 1 met criteria for a principal diagnosis of DMDD, a secondary diagnosis of ADHD-Combined Type, and a third diagnosis of Enuresis. Case 2 also met criteria for a principal diagnosis of DMDD and a secondary diagnosis of ADHD-Combined Type. Case 3 met criteria for a principal diagnosis of DMDD and a secondary diagnosis of Major Depressive Disorder (MDD). All three children were living with their married biological parents who had tertiary degrees and worked full time. All children spoke English as their first language.

Participants were recruited from advertisements in the newsletters of local primary schools. Inclusion criteria included: (i) primary DMDD diagnosis; (ii) no current diagnosis of ASD, intellectual disability, organic brain injury, psychosis, pervasive developmental disorder, vision impairment or physical impairment; (iii) not currently receiving psychological or pharmacological intervention; and (iv) between 7 and 11 years of age. Comorbid secondary anxiety, depressive, externalising, or ADHD diagnoses were not exclusion criteria for this study due to the high incidence of irritability among these disorders (Evans et al., 2017; Eyre et al., 2017; Mulraney et al., 2016).

Seven children were initially screened for suitability based on inclusion criteria prior to attending the clinic for a full assessment. One eligible participant withdrew following the pre-treatment assessment due to commencing another psychological intervention prior to beginning treatment in the present study. A second eligible participant was no longer able to be contacted following the initial screen. Two further participants who were screened were referred to other services due to meeting exclusion criteria (existing ASD diagnoses).

Procedure

Pre-assessment phase

This study was approved by a University Human Research Ethics Committee (2018/281). Verbal informed consent was obtained by parents during an initial telephone screen conducted to assess the presence of irritability and any exclusion criteria. Parents were then provided information and consent forms to return via email if the child did not meet any exclusionary criteria. Next, the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (K-SADS-PL) was conducted over the telephone with the first author and both the parent and child attended an appointment at the University, at which time the child version of the K-SADS-PL was administered in addition to parent- and child-report questionnaires by the first author.

Treatment phase

Treatment was based on an updated version of PST (as reported in Waters et al., 2015). The treatment phase began 2 weeks following the end of the baseline phase, during which time a call was made to the parents to provide them with information on the treatment program and socialise parents to the process. Parents were emailed a treatment booklet which detailed the information outlined in the phone call. During the phone call (30–40 min), parents were informed of the rationale of the treatment approach and how the PST program works, which included instructions on parents sitting with their child to complete the program for the first session so they can understand what their child is learning. Parents were also informed of strategies they could use to promote their child's learning in everyday situations, including reminders of the key learning principles (‘look for good’, ‘look for calm’, ‘use both options’, and ‘never give up’) and asked to complete a monitoring form of their child's engagement during each session of PST to be sent back to the first author at the conclusion of the program. Parents were provided instructions on how to access and download the program onto their home computer which was emailed via link and modified for use on PC and Macintosh computers. The program was downloadable in a Zip file from Google-Drive for ease of use and installation (see Waters et al., 2015 for specific details). Weekly sessions and weekly monitoring phone calls were scheduled during the phone call with parents. Children were also informed during the call about the rationale for the program and were taught the four key learning principals.

Prior to beginning the training, participants listened to instructions via the computer on the necessary strategies they were required to use, including ‘look for good’, ‘look for calm’, ‘use both options’, and ‘never give up’. Participants then began training after completing six random practice trials, by clicking on a happy (e.g., animals, happy people) or calm (e.g., a book, a vase) image in an array of unpleasant images (e.g., a person in hospital or house on fire) in a 3 × 3 or 4 × 4 array. Arrays consisted of one to three positive images (targets) with the remainder as unpleasant background images. Each new trial began immediately after the participant correctly clicked on the target image. Between each block of trials, participants listened to a jingle which restated the strategies they needed to employ (e.g., ‘look for good’, ‘look for calm’, ‘use both options’, and ‘never give up’), after which the participant was then instructed to repeat the jingle out loud to allow for consolidation of these strategies. Arrays were presented in four blocks of calm trials, four blocks of positive trials and one block of both positive and calm trials, containing 20, 26, and 40 trials, respectively. Children also completed one of three short intermission games after blocks two and six (e.g., popping balloons displaying the jingles in the balloons, clicking on happy faces as fast as possible within 30 s, and recalling happy faces among distracting faces). Following each session, children were required to repeat the search strategies out loud and they were recorded via the microphone on the computer. Children then completed satisfaction and learning ratings at the end of each session. The output files containing the verbalisation recordings, child ratings, and the treatment responses (i.e., reaction times during PST) were sent back to the first author automatically via email when the program closed, or manually via email if completing PST offline. The training was completed four times each week for a duration of 3 weeks, totalling 12 sessions (Waters et al., 2013, 2015).

Post-treatment phase

Within 2 weeks of completing the treatment program and at 2-months post-treatment, all measures completed at the initial assessment phase were readministered. Parents completed the K-SADS-PL via the phone and children attended an appointment at the University to complete the child K-SADS-PL, during which time questionnaires were completed again. These interviews were conducted by two independent assessors who were second-year graduate students in clinical psychology trained in administration of the K-SADS-PL and were blinded to the pre-treatment diagnoses.

Outcome Measures

Feasibility

Feasibility was evaluated in terms of treatment completion and the technical and practical access and delivery of the program. Treatment completion was considered feasible if (1) parents completed the treatment phone call and demonstrated an understanding of the requirements of PST, including when their child can complete the program and how they can reinforce their child's key learnings of PST in everyday life, (2) if children completed all 12 sessions of the program (i.e., participant retention across treatment), and (3) if participants were able to complete the 12 sessions within the time frame initially designed (i.e., 4 sessions per week for 3 weeks; Waters et al., 2015). Feasibility with regard to the technical and practical aspects of the program was indexed based on (1) if families were able to access and install the program on their home computer, (2) if children were able to progress through the program without technical problems (e.g., internet access issues), and (3) if the three treatment output files (verbalisations, child engagement ratings, and child treatment responses during PST) were received either automatically or manually sent back to the secure email address at the end of each session or at the end of the program, respectively.

Acceptability

Acceptability was based on (1) child engagement and enjoyment in the program, indexed by child and parent ratings of the sessions (ratings of enthusiasm, concentration, enjoyment, and usefulness), and (2) child learning and engagement during the program, indexed by the number of treatment-relevant verbalisations of the key learnings. Child ratings were provided during PST on-screen at the end of each session with data saved as an output file. The questions included (1) How keen were you to complete the program today? (2) How well could you keep your mind on the program today? (3) How much did you enjoy completing the program today? (4) How useful is the program in helping you to feel happy and calm? Children used a 9-point rating scale (0 = not at all to 8 = very much) for each question. Questions on the monitoring form completed by the parents throughout PST included (1) How keen was your child to complete the program today? (2) How well do you think that your child kept their mind on the program today? (3) How much do you think your child enjoyed doing the program today? (4) How useful do you think the program is in helping your child to feel better? Responses were rated on the same 9-point rating scale used by the children (0 = not at all to 8 = very much) for each question. Child and parent ratings of PST at sessions 1, 6, and 12 were used to demonstrate stability of child engagement and enjoyment across sessions, with higher ratings indicating higher engagement and enjoyment in the program. PST was considered acceptable if stable ratings of each question (enthusiasm, concentration, enjoyment, and usefulness) were reported. The second indicator of acceptability was determined by the number of child verbalisations at the end of the program to determine child engagement and learning during PST. This was indexed both by the number of strategies children verbalised at the end of the sessions, and whether they verbalised non-treatment-related content (Waters et al., 2015, 2016), with more treatment-relevant verbalisations indicating higher engagement and learning. The total number of child verbalisations at sessions 1, 6, and 12 out of the maximum possible in each of these sessions (range = 0 to 4 per session) was used to demonstrate stability in engagement and learning across sessions.

Efficacy

Efficacy of the treatment was determined by changes in the primary and secondary outcome measures outlined below at post-treatment phases compared to pre-treatment and baseline phases.

Primary outcome measures

Diagnostic status

The K-SADS-PL (Kaufman, Birmaher, Brent, & Rao, 1997) was used to assess children's diagnostic status, for which they received a clinician severity rating (CSR) of four or higher (scale 0–8). An initial screen was conducted consisting of a symptom screen for DSM-5 criteria. Children who met threshold scores for diagnostic areas were administered the modules for affective, psychotic, anxiety, behavioural, and neuropsychological disorders. The parent K-SADS was administered over the phone and face-to-face with children. The K-SADS-PL assessments were administered by postgraduate clinical students who had undergone K-SADS-PL training. Independent assessors were used at follow-up and were blind to the child's diagnostic profile at pre-treatment assessments. The outcomes of the interviews were reviewed with the project team during a meeting to arrive at a consensus diagnoses and CSRs. Diagnoses that children met clinical criteria for at pre-treatment were readministered during the baseline and treatment phases (i.e., DMDD, ADHD-Combined).

Irritability symptoms

Participants and their parent or carer were first be administered the 7-item Affective Reactivity Index (ARI; Stringaris et al., 2012) used to assess irritability in clinical and community samples. The ARI consists of both self- and parent-report versions of identical scales measuring three aspects of irritability; the threshold for an angry reaction, the frequency of anger in terms of both feelings and behaviours, and the duration of these feelings (Stringaris et al., 2012). The ARI has six scored symptom items and one impairment item that is not scored. Responses are on a 3-point Likert scale (0 = not true, 1 = somewhat true, 2 = certainly true) yielding a minimum score of zero and a maximum score of 12 based on their prior 6 months (Mulraney, Melvin, & Tonge, 2014). Within a US and UK sample, the parent and child ARI were significantly correlated r = 0.58 and r = 0.73, respectively. Moreover, there was a Cronbach's alpha of 0.92 and 0.88 in the US sample and 0.89 and 0.90 in the UK sample for parental and self-report scales, respectively (Stringaris et al., 2012). Children with a score above four on the child-report ARI, or above three on the parent-report ARI indicate clinical levels of irritability. These measures were readministered throughout the baseline and treatment phases weekly.

Secondary outcome measures

Children's Global Assessment Scale

The Children's Global Assessment Scale (CGAS; Shaffer et al., 1983) is a clinician-rated measure used to assess changes in severity of overall disturbance in functioning from 0 to 100 (81–100 = normal functioning, 61–80 = slight disability, 41–60 moderate disability, 1–40 = serious disability) (Shaffer et al., 1983). The CGAS has demonstrated reliability between raters and across time and has good discriminant and concurrent validity (Dyrborg et al., 2000; Shaffer et al., 1983). This was completed throughout the baseline and treatment phases.

Data Analyses

Data analysis of this single case design study first included visual analysis of the linearly graphed data (Ledford, Lane, & Severini, 2018; Wolfe, Barton, & Meadan, 2019). In addition to visual inspection of the data, the improvement rate difference (IRD) was also calculated for the primary outcome measures. IRD is a non-overlap method used to identify improvement rates from the baseline phase to the treatment phase (Parker, Vannest, & Brown, 2009). Improvement rates in each phase are calculated by the number of ‘improved data’ (overlapping data at baseline phase; non-overlapping data from the treatment) divided by the total data in each of the phases. IRDs are then calculated by subtracting improvement rates for the baseline phase from those of the treatment phase (Parker et al., 2009; Wolfe et al., 2019).

Results

Feasibility

Parents of Case 1, Case 2, and Case 3 completed the treatment phone call and demonstrated an understanding of the requirements of PST by suggesting practical ways to implement the learning strategies at home and scheduling completion times for PST. Regarding participant retention, Case 1 and Case 2 completed all 12 sessions of the treatment program. The parent of Case 3 declined to complete treatment after child refusal to continue after the first session during week 1. The other two children completed all 12 treatment sessions in 3 (Case 1) or 4 (Case 2) weeks. Regarding the technical and practical aspects of the program, Case 1, Case 2, and Case 3 were able to access the treatment program via email and install it on their home computers. There were no internet access issues throughout the program or difficulties progressing through the training. However, some data were not available due to problems with the verbalisation page freezing for both Case 1 (week 3, session 1) and Case 2 (week 2, sessions 1 and 2), and thus, the three verbalisation output files were not available during those sessions.

Acceptability

Table 1 summarises child-reported ratings of enthusiasm to complete the program, their concentration during the program, how enjoyable they found the program, and how useful they thought it was. Ratings of enthusiasm for the program were moderate for Case 1 (‘somewhat’ to ‘quite a bit’ enthusiastic) and were observed to decrease slightly from session 1 to session 6, and remained consistently high for Case 2. Both cases indicated they were able to concentrate on the program ‘quite a bit’, with both cases reporting increases in concentration across sessions. Both Case 1 and Case 2 increased in their ratings of how enjoyable they found the program, with Case 1 indicating they enjoyed it ‘a lot’ and Case 2 indicating they enjoyed it ‘quite a bit’ at session 1 and both reported enjoying it ‘very much’ at the final session. Finally, ratings of usefulness suggested that both cases found PST ‘quite a bit’ to ‘very much’ useful across sessions.

Table 1 Child Ratings of Acceptability of PST across Sessions 1, 6, and 12

Note. Ratings were based on a 0 to 8 scale where 8 = very much.

Parent rating data (see Table 2) was available for Case 2 as Case 1 did not complete the monitoring form throughout the treatment program. Based on parent-report ratings of their child's engagement and enjoyment, Case 2 demonstrated ratings of enthusiasm that were initially very high (‘very much’ enthusiastic) and were observed to decrease from session 1 to session 6 and remained stable (‘somewhat’ enthusiastic). Parent ratings of Case 2 indicated they were able to concentrate on the program to at least a moderate degree, however indicated higher levels of concentration at the first and final sessions. Ratings of how enjoyable Case 2's parent thought their child found the program suggested they enjoyed it ‘a lot’ at session 1, however this decreased to within the moderate range at sessions 6 and 12. Finally, ratings of usefulness suggested that Case 2's parent found PST ‘somewhat’ to ‘quite a bit’ useful across sessions for their child.

Table 2 Parent Ratings of Acceptability of PST across Sessions 1, 6, and 12 for Case 2

Note. Ratings were based on a 0 to 8 scale where 8 = very much.

Due to technical difficulties submitting the recordings of verbalisations during week 2, the number of verbalisations at sessions 1, 6, and 12 were used as an index of child engagement and learning in the program. Case 1 and Case 2 verbalised all four search strategies at sessions 1, 6, and 12, indicating they were able to attend to and recall the key search strategies of the program at the beginning, middle, and end of the program. Neither Case 1 nor Case 2 verbalised any non-treatment-related content during these sessions.

Efficacy

Primary outcome ratings

Diagnostic status

Child diagnostic status across pre-treatment, baseline, treatment, and post-treatment phases are presented in Figure 1. Visual inspection of the plots indicated declines to subclinical levels of DMDD during the treatment phase; however, scores did not decline for Case 1 until the last week of treatment. This was corroborated by IRD scores, suggesting that for Case 1 there was a 42% IRD during the treatment phase. Case 2 demonstrated 100% IRD, with scores demonstrating a clear decline in the treatment phase. Following visual inspection of the post-treatment phase, both participants met clinical criteria (i.e., CSR of 4 or higher) for ODD rather than DMDD. During the post-treatment phase, Case 2 remained stable for ODD scores; however, Case 1 met criteria for DMDD at the final follow-up assessment. Due to the change in diagnostic status to no longer meeting criteria during the post-treatment phase, IRDs were not calculated for the follow-up assessments.

Figure 1. Clinician severity ratings of diagnoses across baseline, treatment, and post-treatment phases.

Additional diagnoses meeting clinical criteria at initial assessment (ADHD-Combined for Cases 1 and 2, and Enuresis for Case 1) were also assessed during the treatment and post-treatment phases to monitor changes over time. Case 1 demonstrated a stable pattern for ADHD-Combined, which remained clinical at post-treatment. However, variability was observed for Case 2, with declines into treatment for the severity of ADHD-Combined, that returned to baseline following the end of treatment. Case 1 also met criteria for Enuresis, which remained stable from the beginning of the baseline phase to the end of treatment, but was not expected to be amenable to change with PST.

Irritability symptoms

Visual inspection of parent-reported irritability for Case 1 (Figure 2) demonstrated variable results, with declines in irritability towards the end of the treatment phase. Of note however, Case 1 demonstrated declines in the baseline phase as well, with a return to high levels of irritability at the beginning of treatment. On the other hand, Case 2 had consistent and stable declines in irritability from the baseline phase until the end of treatment. IRD calculations corroborated these results, whereby Case 1 demonstrated an overall improvement rate difference of 50% during treatment, indicating questionable improvement in irritability symptoms and Case 2 had an IRD of 100% for the treatment phase. Visual inspection of both cases for the post-treatment phase indicated a decline in parent-reported irritability for Case 1, supported by IRD scores indicating a 75% IRD when comparing post-treatment relative to baseline scores. Visual inspection and IRD scores (IRD = 0%) for Case 2 indicated a clear return to baseline for irritability during the post-treatment phase. Overall, there appears to be inconsistent results for parent-reported irritability symptoms, with Case 1 demonstrating large treatment effects at follow-up, and Case 2 having large effects during the treatment phase.

Figure 2. Irritability ratings baseline, treatment, and post-treatment phases based on parent-report.

Observation of ratings of irritability based on child-report (Figure 3) for both cases showed somewhat variable reductions during the treatment phase. Case 1 had a downward trend during the treatment phase, although initial scores for irritability were reportedly low. Case 2 demonstrated some reduction in child-reported irritability in the treatment phase relative to baseline, although had an upward trend. These results were supported with the IRD scores, indicating small (<50%) effects for Case 1, and moderate effects for Case 2 (67%) during the treatment phase. Both cases demonstrated return to baseline scores for child-reported irritability based on both visual analysis and IRD scores (<50%).

Figure 3. Irritability ratings across baseline, treatment, and post-treatment phases based on child-report.

Secondary outcome ratings

Global improvement

The visual examination of global improvement ratings based on the CGAS is presented in Figure 4. Both cases demonstrated improvement across the treatment phase, with upward trends. This was supported by IRDs whereby both cases had moderate effects (67%) during the treatment phase. Case 1 had a slight decline at post-treatment assessment ratings compared to the treatment phase, however had a 100% IRD at follow-up relative to the baseline phase. Case 2 had a return to baseline for their CGAS ratings at follow-up, with an IRD indicating no improvement during this phase.

Figure 4. CGAS ratings across baseline, treatment, and post-treatment phases.

Discussion

Given that models of irritability include biases in attention as a key mechanism maintaining irritability in youth (see Brotman et al., 2017), it has been proposed that treatments of irritability should target these key mechanisms (Brotman et al., 2017; Kircanski et al., 2019). As a result, this study was a first test of the feasibility, acceptability, and efficacy of PST for irritable youth. Based on the efficacy of PST for anxious youth (Waters et al., 2013, 2015), who demonstrate similar underlying threat attention biases (Dudeney et al., 2015; Shi et al., 2019) as those observed in irritable youth (Hommer et al., 2014; Kircanski, White, et al., 2018; Salum et al., 2017), it was hypothesised that PST would also be feasible, acceptable, and efficacious within a sample of irritable youth.

PST was largely feasible within this population, as all three families completed the parent component part of the program and two of the three children who commenced the program completed all 12 sessions. However, one family withdrew from the program during week 1 due to child refusal to continue. As parent accommodation to child refusal to engage in treatment is common among youth presenting with behavioural difficulties (Chacko et al., 2016; Kircanski et al., 2019; Nobel et al., 2020), this may have contributed to Case 3's withdrawal from the program. Another notable difference between Case 3 compared to Cases 1 and 2 was the presence of comorbid MDD. Although youth presenting with clinical anxiety disorders have demonstrated reductions in parent- and child-reported depression symptoms in prior studies (Waters et al., 2015, 2016), it is unclear whether the positively oriented nature of PST impacted on the engagement and willingness of Case 3 to participate given reductions in experiences of pleasure and enjoyment are often observed in MDD (Watson, Harvey, McCabe, & Reynolds, 2020).

Although Case 2 required 4 weeks instead of 3 weeks to complete PST (Waters et al., 2015), the longer time-frame required to complete the program was similar to prior intervention studies utilising PST (Waters et al., 2016). Moreover, despite some technical problems preventing output files being recorded, all participants were able to access and install PST on their home computers, demonstrating ease of access and the practical feasibility for families implementing PST for irritable youth at home.

When assessing acceptability, child ratings of the sessions were stable and high, with ratings of enthusiasm, concentration, enjoyment, and usefulness suggesting that the children found the program to be engaging and enjoyable. Parent ratings for Case 2 indicated similar results, with ratings suggesting at least moderate levels of engagement and enjoyableness. Furthermore, as both children were able to verbalise all four of the strategies learned in both the first, sixth, and final sessions, this suggests that the intervention was targeted at an appropriate and achievable level for this population with regard to their ability to engage with the program and learn the key strategies of PST.

With regards to assessing the efficacy of PST for irritable youth, results were somewhat mixed for the primary outcome ratings. Results of the intervention across treatment appear promising for PST, with both cases no longer meeting clinical criteria for DMDD during treatment. At the post-treatment assessment, both children did have a return to clinical criteria upon initial assessment, however, they met criteria for the less severe diagnosis of ODD rather than DMDD (American Psychological Association, 2000). Importantly, although Case 1 did meet criteria for DMDD again at the 2-month follow-up assessment, indicating a return to clinical criteria, this was at a reduced severity as compared to in the pre-treatment phase.

Inconsistencies were observed in the reductions of ratings for both parent- and child-reported levels of irritability, with large treatment effects seen in the post-treatment assessments for Case 1, and large treatment effects seen in the treatment phase for Case 2. Furthermore, given both children demonstrated stable or increasing patterns of ADHD-Combined symptomology, executive functioning and attentional difficulties were perhaps impacting on treatment efficacy. Taken together, although the efficacy of the primary outcome variables was promising during or immediately after treatment, further work is needed to determine how treatment effects can be maintained at follow-up, such as through the completion of sessions over a longer time period (e.g., booster sessions), or through combining PST with additional strategies to target comorbid diagnoses.

Assessment of the efficacy of PST based on secondary outcome ratings suggested similarly variable results. The improvements in global functioning observed for both cases across the treatment phase demonstrated promise for the effectiveness of PST, however reductions in global functioning at follow-up to levels similar to at the baseline phase further reflect the pattern of impacting attentional difficulties consistent with ADHD.

Taken together, these results suggest some positive treatment indicators for PST within youth with high irritability, as seen in prior studies in anxious populations (Waters et al., 2013, 2015). However, further research is required to address the potential influence of cognitive and behavioural features of comorbid diagnoses. Specifically, some studies investigating treatment effects for disruptive behaviour disorders have found improved treatment effects for sessions completed over longer time periods compared to those completed in a more intensive manner (Comer et al., 2013; Granski, Javdani, Anderson, & Caires, 2020). In addition, given that DMDD and severe irritability are highly comorbid with ADHD (Benarous et al., 2017; Eyre et al., 2017; Mulraney et al., 2016; Waxmonsky et al., 2017) and ODD (Althoff et al., 2016; Evans et al., 2017; Mayes et al., 2015), irritable youth might benefit more from PST combined with additional interventions, such as parent training, to manage behavioural difficulties (i.e., Nobel et al., 2020), consistent with conclusions based on more recent models of irritability (Kircanski et al., 2019).

Despite offering a first test of the feasibility, acceptability, and efficacy of PST for irritable youth, the study has several limitations which need to be considered. First, although single case series designs and multiple baseline approaches are recommended as initial tests of novel interventions (Chambless & Ollendick, 2001; Gallo et al., 2013), this study was a small pilot sample and thus was not designed to perform more complex analyses. Additionally, this study did not allow for comparison to a control group of youth without clinical diagnoses and/or low levels of irritability. Recent findings by Haller et al. (2022) indicated no significant difference when comparing an active group to a control group for interpretation bias training despite prior evidence for its utility (Penton-Voak et al., 2013; Stoddard et al., 2016). Although these studies target different underlying mechanisms in irritability, it highlights the importance of comparison to a control group as the next steps in understanding the clinical efficacy of PST for irritable youth. Finally, given irritability is highly comorbid across internalising and externalising disorders (American Psychological Association, 2000), parsing the effectiveness of PST for irritability alone proves challenging. Additionally, some research has identified that irritability may not be a unitary construct, with tonic (i.e., persistently irritable mood) and phasic (i.e., temper outbursts or tantrums) irritability differentially associated with internalising and externalising disorders in the longer term, respectively (Copeland, Brotman, & Costello, 2015; Silver et al., 2021). Although these forms of irritability are moderately-highly correlated (Silver et al., 2021), with each type associated with the development of the other (Copeland et al., 2015), it may be the case that treatment effects vary based on the presentation of irritability, thus calling for the need to measure irritability based on presentations of tonic and phasic irritability separately. Thus, future work may consider larger-scale trials of PST for irritable youth adapted to include key treatment targets for disruptive behaviour difficulties, comparison to a control condition, and consideration of the role of both tonic and phasic forms of irritability in treatment outcomes.

Overall, this first examination of the feasibility, acceptability, and efficacy of PST for irritable youth provides a foundation for the usefulness of this treatment approach. Further studies that assess the efficacy of PST in larger samples relative to a control condition are required. Furthermore, studies that assess longer treatments with PST and the effectiveness of combining PST with parent training may further add to PST's efficacy in irritable youth.