Introduction
Attention deficit hyperactivity disorder (ADHD) is the commonest disorder presenting to Child and Adolescent Mental Health Services (CAMHS) in Ireland (HSE, 2013). Globally, it is one of the most frequently occurring childhood psychiatric conditions, with prevalence estimates of approximately 5% (Polanczyk et al. Reference Polanczyk, de Lima, Horta, Biederman and Rohde2007). ADHD is characterised by high levels of inattention, hyperactivity and impulsivity that cause significant functional impairment (Thapar & Cooper, Reference Thapar and Cooper2016). Young people (YP) with ADHD often present with comorbid behavioural difficulties including oppositional defiant disorder (ODD) (Jensen & Steinhausen, Reference Jensen and Steinhausen2015) and sleep difficulties (Wajszilber et al. Reference Wajszilber, Santiseban and Gruber2018). ADMiRE (ADHD Assessment, Diagnosis, Management initiation, Research and Education) is the first public specialist ADHD service in Ireland. Set up in September 2018 in South Dublin Linn Dara CAMHS, the primary goal was to reduce waiting times and provide early access to assessment and management for YP with ADHD (McGrath, Reference McGrath2020). Audits within ADMiRE had demonstrated improvements in clinical outcomes, but service user satisfaction had not previously been measured.
Service user satisfaction is considered by many to be a performance measure of the quality of health care (Donabedian, Reference Donabedian1981, Cleary & McNeil, Reference Cleary and McNeil1988, Ygge & Arnetz, Reference Ygge and Arnetz2001), and it has even been suggested that satisfaction should be the only measure of service quality in child health (Kennedy, Reference Kennedy2010). Higher levels of patient satisfaction are associated with improved health outcomes, for example (Greenfield et al. Reference Greenfield, Kaplan and Ware1985, Stewart, Reference Stewart1995, Meterko et al. Reference Meterko, Wright, Lin, Lowy and Cleary2010), better adherence to clinician recommendations (DiMatteo, Reference DiMatteo1994, Zolnierek & DiMatteo, Reference Zolnierek and DiMatteo2009) and positive sequelae for healthcare staff including greater staff satisfaction, improved staff engagement and reduced turnover of staff (Rave et al. Reference Rave, Geyer, Reeder, Ernst, Goldberg and Barnard2003).
While there is widespread recognition of the importance of understanding patient satisfaction in order to ensure services offer genuine patient-centred care, there are a number of difficulties that arise when trying to measure patient satisfaction including the subjectivity of patient ratings and the variability of patient expectation (Crow et al. Reference Crow, Gage, Hampson, Hart, Kimber, Storey and Thomas2002). It is now generally accepted that it is more meaningful to focus on specific measures of patient experience rather than more general measures of satisfaction (Coulter et al. Reference Coulter, Fitzpatrick and Cornwell2009). Consequently, ‘Patient Reported Experience Measures’ (PREMs) are now widely used to measure satisfaction (Bull et al. Reference Bull, Byrnes, Hettiarachchi and Downes2019).
The Experience of Service Questionnaire (ESQ) is a PREM that was developed by the UK Commission for Health Improvement for measuring service satisfaction specifically in child and adolescent mental health services (CAMHS) (Attride-Stirling, Reference Attride-Stirling2003). It is used as the core measure of service experience in at least 20% of CAMHS in the UK (Barnes et al. Reference Barnes, Devanney, Uglebjerg, Wistow and Hartley2010) and shows robust ability to assess satisfaction with care (SWC) (Brown et al. Reference Brown, Ford, Deighton and Wolpert2014). Unlike many other PREMs that have been used internationally in CAMHS, the psychometric properties of the ESQ have been established and a scoring protocol and norm-referenced tables have been developed (Brown et al. Reference Brown, Ford, Deighton and Wolpert2014). Other advantages of the ESQ are that it is free to use, parent and youth-report versions are available and it is quick to complete. YP were involved in the development of the ESQ, therefore it includes constructs that are important to them; this represents a change to much other related research (Davison et al. Reference Davison, Zamperoni and Stain2017). Finally, the ESQ includes both quantitative and qualitative components allowing users to record in-depth information about the factors that are most important to them.
The UK Child Outcomes Research Consortium recommends that the ESQ is used in conjunction with symptom severity measures (CORC, 2019). Symptom severity has been identified as a potential determinant of parent and YP satisfaction in child mental health services and it is important to understand the reported service user experience alongside the YP’s symptom severity. A number of studies have demonstrated a strong relationship between the YP’s symptoms and reported levels of satisfaction, with lower satisfaction associated with higher symptom load (Carscaddon et al. Reference Carscaddon, George and Wells1990, Godley et al. Reference Godley, Fiedler and Funk1998, Garland et al. Reference Garland, Aarons, Saltzman and Kruse2000, Solberg et al. Reference Solberg, Larsson and Jozefiak2015), and the presence of disruptive or externalising behaviours (Bjørngaard et al. Reference Bjørngaard, Andersson, Ose and Hanssen-Bauer2008, Turchik et al. Reference Turchik, Karpenko, Ogles, Demireva and Probst2010). Other studies however have reported no association between the YP’s symptoms and parent/youth satisfaction (Stuntzer-Gibson et al. Reference Stuntzer-Gibson, Koren and DeChillo1995) or a mix of findings, either with symptom severity associated with parent but not youth satisfaction (Garland et al. Reference Garland, Haine and Boxmeyer2007, Bjørngaard et al. Reference Bjørngaard, Andersson, Ose and Hanssen-Bauer2008) or conversely symptom severity associated with youth but not parent satisfaction (Garland et al. Reference Garland, Aarons, Hawley and Hough2003). This inconsistency in study results is likely to stem in part from heterogeneity of study methods; very few studies use the same measure and there is variation in respondents with some studies collecting information from parents/carers, others from youth, and others from both parents and youth.
In Ireland there is a dearth of research relating to service user satisfaction with CAMHS, with a recent systematic review identifying only seven studies (Leahy & McNicholas, Reference Leahy and McNicholas2021). None of these studies concurrently measured symptom severity in the YP, and therefore were unable to specifically investigate the impact of symptom severity on parent/YP satisfaction.
There were three objectives of this study. Given the strong evidence in the literature for differing perspectives of parent and youth, the first objective was to use a validated PREM to quantitatively evaluate both parent/carer and YP satisfaction with ADMiRE. In order to better understand clinical factors that influence satisfaction, the second objective was to investigate associations between clinical severity and reported satisfaction with the service. Finally, service user feedback is often considered quantitatively as a summary score, and does not typically provide information about the particular aspects of care that drive satisfaction for the service user (Brown et al. Reference Brown, Ford, Deighton and Wolpert2014). To try to better understand factors that influence service user satisfaction, the third objective of this study was to conduct an exploratory qualitative analysis to identify factors that are most important to service users in ADMiRE.
Methods
Participants
All parents/carers and YP 9–18 years who attended a routine, face-to-face clinical appointment in ADMiRE were invited to participate. Data collection took place in two phases: February–early March 2020 (referred to as pre-COVID) and September–December 2020 (referred to as post-COVID). It was not possible to collect data between March and September 2020 due to COVID-19 restrictions.
Procedure and instruments
Prior to clinical review, all parents/carers completed the ADMiRE symptom rating scale, comprised of the 26-item SNAP-IV (Swanson, Reference Swanson1995), which measures ADHD and ODD severity, and the Children’s Sleep Habits Questionnaire (CSHQ) (Owens et al. Reference Owens, Spirito and McGuinn2000), a 33-item multiple-choice parent-report questionnaire for childhood sleep problems.
Following the clinical review, participants (parents/carers and YP > 9) completed the ESQ (Attride-Stirling, Reference Attride-Stirling2003, CORC, 2019) anonymously. The ESQ consists of 12 quantitative items, rated on a 3-point Likert scale, and three qualitative free-text sections.
Statistical analysis
Data were analysed in the SPSS Statistics software package. Individual instrument scores were calculated and plotted to visualise their distributions. SNAP-IV and CSHQ data followed a normal distribution. Scores in satisfaction studies are typically inflated and the data skewed towards high values. Exploring descriptive statistics and visualising plots suggested that the ESQ data in this study did not follow a normal distribution and for this reason non-parametric statistical tests were used.
Analysis of service user SWC
SWC was calculated from summing scores from questions 1, 2, 3, 4, 5, 6, 7, 11 and 12 of the ESQ, giving a minimum score of 0 and a maximum score of 18. The raw data recorded in the ESQ is ordinal; but has been transformed into a standardised interval scale using an Item Response Theory (IRT) model (Brown et al. Reference Brown, Ford, Deighton and Wolpert2014). A conversion table allows users to translate the hand-scored ESQ into population-referenced IRT scores for SWC. In this study, raw ESQ scores are converted into IRT scores for analysis. When the ESQ score (0–18) is transformed using IRT, the range of the standardised interval score is from −3.10 to 0.61; see (Brown et al. Reference Brown, Ford, Deighton and Wolpert2014).
IRT scores for parent/carer-reported and youth-reported IRT were calculated. As there were two phases of data collection, Mann-Whitney U tests were performed to investigate differences in SWC pre- and post-COVID-19. A Wilcoxon signed rank test was used to investigate whether there was a difference between parent and YP SWC scores. The relationship between parent/carer- and youth-reported SWC was explored using Spearman’s correlation.
Analysis of symptom severity and impact of severity on SWC
For SNAP-IV data, average scores were calculated for inattention, hyperactive/impulsive, and ODD subscales and compared with established cut-off scores above the 95th percentile (1.78 for inattention, 1.44 for hyperactive/impulsive and 1.88 for ODD subscales) (Swanson et al. Reference Swanson, Kraemer, Hinshaw, Arnold, Conners, Abikoff, Clevenger, Davies, Elliott and Greenhill2001). A cut-off total CSHQ score of 41 was used to define sleep disorder (Owens et al. Reference Owens, Spirito and McGuinn2000). Differences in pre- and post-COVID symptom severity were investigated for each category.
Mann-Whitney U tests were performed to investigate differences in satisfaction scores between high and low symptom severity (severity of ADHD, ODD and sleep disturbance). These were calculated separately for parent and child satisfaction scores. The relationship between ADHD symptoms and satisfaction was explored using Spearman’s correlation.
Exploratory qualitative analysis
Parent/carer and YP responses to the three free-text questions from the ESQ were coded using thematic analysis (Silverman, Reference Silverman2013) in an exploratory qualitative analysis.
Results
67 parents/carers (n = 28 pre-COVID, n = 39 post-COVID) and 44 YP (n = 19 pre-COVID, n = 25 post-COVID) completed the ESQ. Fewer YP completed the ESQ as some were <9 years, some declined to participate, and in some cases parents/carers attended alone. 67 SNAP-IV rating scales (n = 28 pre-COVID, n = 39 post-COVID) and 60 CSHQ sleep rating scales (n = 23 pre-COVID, n = 37 post-COVID) were completed by parents/carers. The YP’s symptoms are based on the parent/carer reports.
SWC
83.5% of parents/carers provided maximum scores on the ESQ, with a median IRT SWC score of 0.61 (range −1.6 to 0.61). 37% of YP gave maximum ESQ scores with a median IRT SWC score of 0.61 (range −1.49 to 0.61).
Difference in SWC pre- and post-COVID-19
There were no statistically significant differences in the pre- and post-COVID data collection phases for parent/carer or for YP SWC scores (parent/carer: median pre-COVID: 0.61, median post-COVID: 0.61; Mann-Whitney U = 532.5, p = 0.79), (YP: median pre-COVID: 0.61, median post-COVID: 0.61; Mann-Whitney U = 217.5, p = 0.59).
Comparison between parent/carer and YP SWC
Parents/carers were significantly more satisfied than YP with care in ADMiRE (Wilcoxon Signed Ranks test, z = −2.19, p = .028). There was a only weak positive correlation in this study between parent/carer and YP IRT SWC scores (Spearman r s = 0.11, p = 0.47).
Analysis of symptom severity
High and low severity groups were generated for each of the four symptom categories, hyperactive/impulsive symptoms, inattentive symptoms, ODD and sleep disorder. Those above established 95th percentile cut-off values were included in the high severity group (Table 1).
Table 1. Numbers of YP in the high and low symptom severity groups for each symptom category
Symptom severity pre- and post-COVID
There were no significant differences in symptom severity pre- and post-COVID-19 (Table 2, Fig. 1)
Fig. 1. Plots showing means of each category of symptom severity pre- and post-COVID-19.
Table 2. Results of analysis investigating differences between means of symptom severity subscales pre- and post-COVID-19 (N = number, SD = standard deviation, p = p-value, CI = 95% confidence interval for the difference in means)
Impact of ADHD, ODD and sleep disorder severity on parent/carer and YP SWC
For parents/carers, there were no significant differences between their levels of SWC for YP with high symptom severity compared with low symptom severity in the four symptom categories that were investigated (Table 3), and there was no statistically significant correlation between symptom severity measures and SWC.
Table 3. Results of Mann-Whitney U analysis investigating impact of symptom severity on parent/carer SWC
*Denotes statistical significance p < 0.05.
YP with severe hyperactive/impulsive and inattentive ADHD symptoms were significantly less satisfied with care than those with less severe ADHD symptoms (Table 3). There was a significant negative correlation between ADHD symptom severity and YP satisfaction [ADHD hyperactive/impulsive symptoms (Spearman r s (42) = −.355, p = .018), ADHD inattentive symptoms (Spearman r s = −.419, p = .005)]. The severity of ODD and sleep disorder symptoms did not impact significantly on YP SWC (Table 3).
Qualitative data analysis
52 of 67 (78%) parents/carers and 28 of 44 (63%) YP provided written responses in the free-text sections of the ESQ. As there were significant differences between parent/carer and YP SWC scores, responses were analysed separately for parents/carers and YP in order to get a better understanding of factors that influence satisfaction in each group.
Two primary themes emerged from the exploratory qualitative analysis. The first theme related to positive aspects of care and the second related to improvements needed in the service (Fig. 2 and Table 4).
Fig. 2. Bar chart showing the percentage of respondents for each of the key themes in the qualitative analysis (hatched bars reflect positive aspects of care, dotted bars reflect areas where improvement is required).
Table 4. Parent/carer and YP’s comments relating to key themes that emerged
Discussion
This study used a quantitative and qualitative approach to determine levels of parent and YP satisfaction with a specialist child and adolescent mental health service. It is the first Irish study to specifically investigate the impact of symptom severity on service user satisfaction.
There were three research questions that were addressed; all focused on developing a better understanding of service user satisfaction. The first question was to determine whether service users were satisfied with the care they receive in ADMiRE. Parent/carer satisfaction data were highly skewed, with high satisfaction levels reported by parents/carers. One difficulty with using PREMs to measure service user satisfaction is that scores are typically inflated, giving an ‘overly positive picture of experiences’ (Crow et al. Reference Crow, Gage, Hampson, Hart, Kimber, Storey and Thomas2002, Brown et al. Reference Brown, Ford, Deighton and Wolpert2014). This may reflect a desire by raters to please the people who are being rated or may result from biased sampling, where only the people who are most satisfied return questionnaires (Brannan et al. Reference Brannan, Sonnichsen and Heflinger1996).
The high level of parent/carer satisfaction in ADMiRE is in line with the results of other studies measuring satisfaction in child mental health services (Garland et al. Reference Garland, Haine and Boxmeyer2007), but is higher than the population-based norms for parent/carer satisfaction that have been established specifically for the ESQ. These norms, based on data from >7000 parents/carers across 41 UK CAMHS, show that only 58.6% of the population gave maximum ESQ satisfaction scores (Brown et al. Reference Brown, Ford, Deighton and Wolpert2014), compared with the 83.5% in this study. Families understood that their data was anonymous and a protocol was in place to ensure that they were reassured about anonymity in order to encourage them to provide genuine feedback about the service. Nonetheless it is possible that parent/carer participants provided inflated results for the reasons described above.
Although SWC was slightly lower for YP compared with parents/carers, 37% of YP attending the service gave maximum ESQ scores. This figure is almost identical to the population-based norms derived from ESQ data collected in UK CAMHS, where 36.5% of YP in UK CAMHS gave maximum ESQ scores (Brown et al. Reference Brown, Ford, Deighton and Wolpert2014).
In line with previous literature (Stuntzer-Gibson et al. Reference Stuntzer-Gibson, Koren and DeChillo1995, Martin et al. Reference Martin, Petr and Kapp2003, Copeland et al. Reference Copeland, Koeske and Greeno2004, Garland et al. Reference Garland, Haine and Boxmeyer2007, Aarons et al. Reference Aarons, Covert, Skriner, Green, Marto, Garland and Landsverk2010, Turchik et al. Reference Turchik, Karpenko, Ogles, Demireva and Probst2010, Athay & Bickman, Reference Athay and Bickman2012, Solberg et al. Reference Solberg, Larsson and Jozefiak2015), there was only low agreement between parent and YP satisfaction ratings, a finding that highlighted the different perspectives that the two groups have. It has been suggested that only the parent/carer ESQ-SWC scores should be used for between-service comparisons, as YP SWC scores are not as reliable as parent/carer scores (Brown et al. Reference Brown, Ford, Deighton and Wolpert2014). However, given the poor consensus between parent and YP report, it is important to consider feedback from the YP in order to develop an understanding of the service-related factors that are important to them, factors that are key to guiding genuine patient-centred service improvement initiatives (Solberg et al. Reference Solberg, Larsson and Jozefiak2015).
The second question investigated the impact of symptom severity on the level of reported satisfaction. While it was hypothesised that increased severity would be associated with reduced satisfaction in parents and YP, the results from this study suggest that the severity of ADHD symptoms, oppositional behaviour or sleep difficulties does not impact significantly on parental satisfaction. From a clinical perspective, this was surprising as parents and carers often present in distress to the service as a result of the severity of these symptoms, and it would make sense that if clinicians were not able to reduce symptom severity, that parents/carers would be dissatisfied.
Previous literature has however been quite mixed, with some studies reporting an inverse relationship between mental health symptom severity and satisfaction (Carscaddon et al. Reference Carscaddon, George and Wells1990, Godley et al. Reference Godley, Fiedler and Funk1998, Garland et al. Reference Garland, Aarons, Saltzman and Kruse2000, Barber et al. Reference Barber, Tischler and Healy2006), but other studies reporting no association (Stuntzer-Gibson et al. Reference Stuntzer-Gibson, Koren and DeChillo1995, Lambert et al. Reference Lambert, Salzer and Bickman1998, Garland et al. Reference Garland, Haine and Boxmeyer2007, Turchik et al. Reference Turchik, Karpenko, Ogles, Demireva and Probst2010), and further well-designed research is required.
While symptom severity did not influence parental satisfaction, it did impact on youth-reported satisfaction. For YP, higher severity of ADHD symptoms was significantly associated with a reduction in satisfaction. This finding was also slightly surprising. The majority of YP are brought to ADMiRE by concerned parents or carers, and outwardly they often show little interest in the service or insight into their symptoms. Although there is no clear pattern of findings from the literature, previous studies have reported an inverse relationship between symptoms and satisfaction (Godley et al. Reference Godley, Fiedler and Funk1998), similar to the results of this study. These results demonstrate the difference between the experience of the parent and YP, and draw attention to the need to collect information from both.
The lack of association between symptom severity and parental satisfaction in this study suggested that other factors are more important determinants of parent-reported satisfaction. Results of the exploratory qualitative analysis highlighted the importance of the therapeutic relationship, with parents/carers commenting spontaneously on how they valued ‘friendly, helpful, supportive staff’ who ‘listened to’ them, took ‘concerns…seriously’ and were ‘good at explaining’, while the key factor that was important to YP was ‘being listened to’. Our results are consistent with others’ analysis of qualitative data relating to service user satisfaction. The value placed by mental health service users on the establishment of the therapeutic alliance, good relationships with clinicians and communication has been highlighted in numerous studies (e.g. Crow et al. Reference Crow, Gage, Hampson, Hart, Kimber, Storey and Thomas2002, McCabe & Priebe, Reference McCabe and Priebe2004, Hart et al. Reference Hart, Saunders and Thomas2005, Davison et al. Reference Davison, Zamperoni and Stain2017).
Facilities in ADMiRE and poor initial access to the service were identified as factors that negatively influenced service user satisfaction. 83% of parent/carer respondents reported that there were no improvements needed in the service, but 6% (3 of 52) commented on initial difficulties with access to the service. ADMiRE is a tertiary level specialist service accepting those with a primary presenting difficulty suggestive of ADHD. A key goal of the service is to provide early access to assessment for ADHD, but if ADHD assessment is not indicated in the first instance (e.g. if the difficulties appear to be related to an anxiety disorder/autism/attachment disorder), the young person will attend for a generic CAMHS assessment and this can cause frustration for parents/carers. 7% of YP expressed dissatisfaction with facilities. On review of the specific comments, the concerns appeared to be related to COVID-19 restrictions that were in place at the time; the need to wear masks (as per government guidelines), a cold waiting area (windows were open for ventilation), and a delay in answering the door (there was a strict policy about numbers in the waiting area).
The findings from this study lead to a number of recommendations relating to person-centred care, shared decision-making and overall engagement with services. There is growing emphasis on the importance of person-centred healthcare, where the patient is central to the care process (de Silva, Reference de Silva2014). To provide genuine person-centred care, it is essential to understand the perspectives and experiences of service users. In line with previous research, data from this study suggests that the perspectives of parents and youth differ significantly, and leads to a recommendation that future satisfaction research in child mental health services always includes input from both parents/carers and YP.
For care to be genuinely person-centred, the clinician needs to give the service user the opportunity to have input into decisions about their care. Such ‘shared decision-making’ is only possible when the clinicians understand the experiences and preferences of service users. There is often a ‘breathtaking gap’ between what the clinician thinks the service user wants, and what they actually want (Coulter, Reference Coulter2012). The finding of an inverse relationship between parent-reported symptom severity and YP-reported satisfaction has important implications for both clinical practice and future research and leads to two recommendations. Firstly, from a clinical perspective, it is important to regularly incorporate the YP’s views of their symptom severity, improve psychoeducation about ADHD for the YP, and generate realistic expectations about available interventions. Secondly, this study considered only parent-reported symptoms, and future research is needed to examine the association between youth-reported symptoms and youth satisfaction.
In relation to engagement with services, this study has clearly demonstrated that factors associated with the development of a good therapeutic alliance are most highly valued by both parents and youth. Clinicians tend to assume that if treatment is not effective in reducing the YP’s symptoms, families will be dissatisfied. In contrast, this study shows that symptom severity does not influence parent-reported satisfaction, but ‘being listened to’ by ‘helpful and supportive staff’ was consistently reported as the factor ‘that was really good about … care’. A fundamental clinical implication of this is that finding the ‘perfect’ pharmacological treatment for ADHD is not necessarily the key to genuine parental satisfaction. The importance of listening to, engaging with, and supporting families should not be underestimated (Carr-Fanning and McGuckin, Reference Carr-Fanning and Mc Guckin2018).
It is also important to consider the strong inverse relationship between symptom severity and satisfaction in YP participating in this study. There are unique challenges for engagement of YP, particularly adolescents, in mental health services. These include the YP striving for independence, rebellion against authority figures and being brought by parents to the service against their will (Turchik et al. Reference Turchik, Karpenko, Ogles, Demireva and Probst2010). If clinicians do not ‘get it right’ for the YP, disengagement is highly likely, indeed 40–60% of adolescents drop out of therapy prematurely (Wierzbicki & Pekarik, Reference Wierzbicki and Pekarik1993). But it is difficult to ‘get it right’ without a deeper understanding of the factors influencing satisfaction for the YP. While the current study has touched on some of the factors influencing youth-reported satisfaction, more in-depth qualitative research is needed to genuinely understand how to better engage the youth attending these services (McNicholas et al. Reference McNicholas, Reulbach, Hanrahan and Sakar2016). There are two recommendations related to this point. Firstly, youth opinion should be sought in all studies aiming to improve child mental health services. Secondly, qualitative research investigating factors influencing satisfaction with mental health services in youth is urgently needed.
Strengths
This study addresses key limitations that have been identified in previous studies of satisfaction in CAMHS. Firstly, the measure of service user satisfaction, the ESQ, is a PREM with established psychometrics and validated population-based norms. Secondly, the ESQ not only allows quantitative measurement of satisfaction, but the three free-text questions also provide qualitative information about the service user’s experience. Thirdly, parents and YP were invited to participate in this study. Much healthcare satisfaction research only records adult data but this precludes investigation of factors that influence YP satisfaction. Finally, this study adopted an innovative approach by simultaneously collecting information about symptom severity and satisfaction. This approach allows quantitative investigation of mental health related factors that may directly influence SWC.
Limitations
The requirement for face-to-face review prior to survey completion limited the sample size, as there was a significant reduction of in-person appointments due to COVID-19. Prior to the pandemic (February–early March 2020), data was collected from all who attended ADMiRE for their scheduled appointments. During the pandemic (Sept–Dec 2020), face-to-face appointments were only being offered when essential (e.g. for required physical monitoring), and many patients were offered online/phone consultation. It is possible that by only collecting data from those attending the service in person during the pandemic, that a sampling bias was introduced. If this were the case, we might expect to see a difference in overall satisfaction with care between the two study periods, however there was no statistically significant change in satisfaction with care after COVID-19 restrictions were introduced in ADMiRE. In addition, when data was being collected during the pandemic (Sept–Dec 2020), appointment times were kept to a minimum and some families were not invited to complete the ESQ if they had had a lengthy clinical review. Assuming a positive association between symptom severity and the duration of clinical review, it is possible that service users with more severe symptoms were not included in the study. If this were the case we would expect differences in symptom severity pre- and post-COVID, however analysis of the symptom severity data showed that this was not the case; there were no significant differences in symptom severity in the pre- and post-COVID-19 groups.
The majority of the children who did not participate were <9 years old, therefore not eligible for study inclusion, but a small number declined to participate, or the parent/carer attended alone. It is possible that those who did not agree to participate or who did not accompany their parent to the appointment may have been less satisfied with the service or had a different level of symptom severity. While it was not possible to determine whether there were systematic differences in satisfaction with care between YP who did/did not participate, there were no statistically significant differences in parent-reported symptom severity between YP who participated and those who did not.
A further limitation was that symptom severity ratings were collected only from parents/carers. It was therefore not possible to examine the relationship between symptom severity reported by the YP and levels of satisfaction.
ADMiRE is a novel, specialist ADHD service and it could be argued that findings from this study may not be generalisable to generic CAMHS settings. The primary aims of this study were firstly to investigate whether parents/carers and YP were satisfied with this new model of care, and secondly to investigate factors that influenced service user satisfaction. While the study did find that parent/carer satisfaction in ADMiRE is higher than population-based norms for parent/carer satisfaction that have been established for the ESQ, the factors that influence satisfaction are consistent with previous analyses of qualitative data relating to service user satisfaction and we would suggest that these are highly generalisable. The establishment of a good therapeutic alliance, attempts to reduce symptom severity, and making service users feel ‘listened to’ are all factors that are attainable across all healthcare services.
Conclusion
This is the first Irish study that has directly investigated the impact of symptom severity on service user satisfaction in a child mental health service. Levels of service user satisfaction were high, however there were significant differences between the perspectives of parents and YP. Greater symptom severity was negatively correlated with youth-reported satisfaction, but did not influence parental satisfaction. Factors related to the therapeutic alliance were identified as being particularly important to both parents and youth. There is an urgent need for further research investigating factors that influence service user satisfaction and it is vital that these factors are considered in the future development of our child mental health services.
Acknowledgements
The authors thank all the families who participated in this study.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflicts of interest
Authors have no conflicts of interest to disclose.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committee on human experimentation with the Helsinki Declaration of 1975, as revised in 2008. This study received ethical approval from the Health Services Executive (HSE) Linn Dara Research Ethics Committee in January 2020. Written informed consent was obtained from all participants.
