Prevention of Emotional Problems in Spanish Schoolchildren: Effectiveness of the Super Skills Multimedia Program to Promote Social-Emotional Skills

Abstract

Compared to traditional psychosocial interventions, the transdiagnostic approach focuses on common processes or factors that contribute to the development and maintenance of emotional problems. In Spain, the Super Skills Program has proven its effectiveness, but issues with displacement or scheduling for families can hinder access to the intervention. This study aims to examine the program’s effectiveness in preventing children’s psychosocial problems through learning socio-emotional skills when self-administered via the Internet. Participants were 283 Spanish children between the ages of 8 and 12. The children and their parents completed anxiety and depression scales before and after receiving the intervention and 12 months later. The program consisted of 8 self-applied sessions, at a rate of two per week, which included emotional education, cognitive restructuring, self-regulation techniques, training in social skills, problem-solving, and video feedback with cognitive preparation. Significant differences in anxiety and depression symptoms were found after the intervention. A significant reduction in anxiety scores and a marginally significant decrease in depression scores was observed at the long-term follow-up. The “intention to treat” analysis revealed a tendency for children who improved to drop out of the program, completing the intervention children with more symptomathology. Learning socio-emotional skills through a self-applied intervention via the Internet effectively reduced emotional symptoms in school children and is a valuable resource to improve child well-being and prevent future psychological problems.

Anxiety problems are very common in childhood, with an estimated percentage of between 12 and 47% of Spanish children showing symptoms (Canals et al., 2019). Anxiety co-occurs with depressive symptoms in approximately 82% of cases (Romero et al., 2010), which means that the prognosis and response to treatment are worse because the symptoms of one problem exacerbate the other (Melton et al., 2016). The high comorbidity between anxiety and depression can be explained by the fact that the two problems share common processes or developmental and maintenance factors, with anxiety often preceding depression (Ghandour et al., 2019; Muris et al., 2017). Anxiety typically begins early, around the age of 6, while the onset of depression occurs later, between the ages of 13 and 20 (Merikangas et al., 2010; Solmi et al., 2022). The average age of anxiety onset coincides with the start of middle childhood, which spans from 6 to 12 years old, a developmental period during which mandatory schooling takes place, posing a significant challenge and an important event in children’s socioemotional development.

Emotional problems that go untreated tend to become chronic and pose a risk factor for other issues in adolescence and adulthood (Essau et al., 2014). Although cognitive-behavioral therapy is the treatment of choice for emotional problems, it does not achieve the expected efficacy in 30 to 50% of cases (Essau et al., 2012; Seligman & Ollendick, 2011). This is likely because interventions often target the patient’s primary problem specifically, neglecting comorbid problems that go untreated (Essau et al., 2014). Positive results are therefore obtained for the primary problem, but the effects on the comorbid problem are small (Garber et al., 2016). The high comorbidity of anxiety with other emotional problems makes it necessary for interventions to address all of the patient’s problems to enhance their moderate effectiveness. In contrast to the traditional therapeutic approach that focuses solely on the primary problem, the transdiagnostic approach is an alternative to consider because it applies a single protocol targeting the common processes or factors that contribute to the development and maintenance of emotional problems.

In Spain, we have two protocolized interventions for child and adolescent emotional problems that follow a transdiagnostic approach. One of them is the Unified Protocol for Children (UP-C; Ehrenreich-May et al., 2018), which has been translated into Spanish, but no data are available on its efficacy in the Spanish child population. The other available intervention is the Super Skills for Life program (Essau & Ollendick, 2013), originally created for the Anglo-Saxon population and adapted for Spanish children aged 6 to 12 with the name Super Skills program (SS; Orgilés, Espada, et al., 2023). The program consists of eight sessions and includes components such as emotional education, cognitive restructuring, relaxation and breathing, social skills training, problem-solving, and video feedback with cognitive preparation. It is a protocolized intervention to teach socioemotional skills using cognitive-behavioral therapy techniques and a transdiagnostic approach. It can be applied in a group or individually, both at school or in a clinical context.

The efficacy of the Super Skills program has been tested in various studies, yielding positive results in Spanish children aged 6 to 12 when applied either in a group or individually, as an indicated prevention program for emotional problems and as a treatment for children with a diagnosis of anxiety disorder and comorbid problems, maintaining long-term effects (Diego et al., 2023; Fernández-Martínez et al., 2019; Fernández-Martínez, Orgilés, et al., 2020; Fernández-Martínez, 2021; Melero, Morales, et al., 2021; Orgilés et al., 2019; Orgilés et al., 2020). Despite the abundant empirical evidence supporting the program, some children may find it difficult to benefit from psychological interventions due to the interference of other activities, transportation issues, or scheduling conflicts with their parents. Considering technological advances, internet-based programs can be an alternative to make interventions more accessible. They provide greater flexibility and have a better cost-effectiveness ratio, as more children can receive the intervention at a lower cost. Furthermore, as children are digital natives and are familiar with technology, multimedia formats appeal to them and promote therapy acceptance (March et al., 2018). Previous studies have shown that self-administered internet-based cognitive-behavioral therapy programs are as effective as in-person therapy for preventing and treating psychosocial problems in children and adolescents (e.g., Spence et al., 2011).

Leveraging the advantages offered by digital resources and the Internet and considering the effectiveness that the Super Skills program has demonstrated in Spanish children, it was deemed appropriate to adapt the program for online implementation. The Super Skills Multimedia program retains the same objectives and content as the original program but is delivered via the Internet. It is self-administered by the child with the support of their parents, who have a written guide to assist their children in activities when needed, and the supervision of a therapist who is available by phone and email to address any issues that may arise during implementation. The efficacy of the multimedia program has been tested in a controlled trial with a group of 75 children with emotional symptoms who were randomly assigned to the intervention group or a control group, showing that those who had received the program had fewer symptoms after its completion (Orgilés, Morales, et al., 2023). However, there is no evidence of its efficacy in preventing anxiety and/or depression problems and promoting socioemotional skills to help children cope with challenges in their daily lives. Therefore, this study aims to: (a) Examine in a community sample of Spanish children aged 8 to 12 whether the self-administered internet-based version of the program (Super Skills Program Multimedia) improves children’s socioemotional competencies, reducing anxiety and depression symptoms after the intervention, according to self-reports of participants and their parents, and (b) determine whether the benefits achieved by the program are stable and maintained over time at a 12-month follow-up.

Method

Participants

This study employed a quasi-experimental design with a single group, pre-test, post-test, and a one-year follow-up. The initial assessment involved 283 Spanish children aged between 8 and 12 (M = 9.56, SD = 1.27) and one of their parents (Table 1). Most of the assessed parents were mothers (88.7%), and 51% of the children were male. More than half of the parents were between 35 and 44 years old, and most were either married or living with their partner (78.1%). Approximately half of the parents reported having full-time employment (53.7%), and a similar proportion had a Degree or Bachelor’s level of education (45.9%). No statistically significant differences were found in the sociodemographic variables between boys and girls, except for the children’s age (a categorical variable) and the type of parental informant. A higher proportion of boys were 9 years old compared to girls (25.9% vs. 15.9%). Regarding the informants, girls were more often assessed by their mothers than by their fathers (83.3% vs. 94.2%).

Table 1. Description of the Sample at Pre-test (N = 283)

Note. M = mean; SD = standard deviation.

^a χ² for categorical variables and Student’s t-test for continuous variables.

^b Cohen’s d for quantitative variables, Cramer’s V for categorical variables, and OR (95% CI) for dichotomous variables.

*p ≤ .05. **p ≤ .01. ***p ≤ .001.

The final sample comprised 33 children who responded to the evaluation one year after participating in the program. Slightly more than half were boys (60.6%), with an average age of 9.19 years (SD = 1.25). The majority of their parents were married (75.8%). About half of the parents were employed full-time (48.5%), and a similar percentage held a degree or bachelor’s level of education (48.5%). Most of the assessed parents were mothers (78.8%).

Instruments

Self-Reports

Brief Child Anxiety Scale (SCAS-C-8; Reardon et al., 2018). This self-report instrument assesses anxiety symptoms in children aged 8 to 12 based on criteria proposed in the Diagnostic and Statistical Manual of Mental Disorders (5^th Ed.; DSM-5; American Psychiatric Association, 2013), and it was adapted in Spain by Rodríguez-Menchón et al. (2021). It consists of 8 items that assess generalized anxiety, separation anxiety, and panic/agoraphobia. Items are rated on a 4-point Likert scale ranging from 1 (never) to 4 (always). The total score is obtained by summing all the items, with higher scores indicating a greater presence of anxiety symptoms. The instrument’s reliability in this sample was adequate (ordinal α = .82).

Short Mood and Feelings Questionnaire (SMFQ; Angold et al., 1995). The SMFQ is a self-report questionnaire that assesses depression symptoms in children. It comprises 13 items rated on a 3-point scale ranging from 0 (not true) to 2 (true). A total score is obtained by summing all the items, with higher scores indicating greater symptom severity. The Spanish version of the SMFQ, which was validated for children aged 8 to 12, was used in this study (Espada et al., 2022). The reliability of the SMFQ in this study was excellent (ordinal α = .92).

Parent Reports

Parent Version of the Spence Children’s Anxiety Scale (SCAS-P-8, Reardon et al., 2018): This is a shortened version of the original SCAS for parents, which provides an overall score of child anxiety from the parents’ perspective. It consists of 8 items rated on a 4-point Likert scale (0 = never to 3 = always), similar to the self-report version, and has demonstrated good psychometric properties. Higher scores indicate higher levels of anxiety in children. In this study, the Spanish-adapted version of SCAS-P-8 (Orgilés et al., 2022) was used, showing adequate reliability (ordinal α = .85).

Parent Version of the Short Mood and Feelings Questionnaire (SMFQ-P; Angold et al., 1995). This is the parent-reported version of the SMFQ and consists of 13 items measuring depression symptoms in children. Parents rate items on a 3-point scale (0 = not true to 2 = always true). The sum of all item scores provides a total depression score, with higher scores indicating higher levels of depression symptoms in children. This instrument has shown good properties for use with Spanish children (Fernández-Martínez, Morales, et al., 2020). In this study, the reliability of the total SMFQ-P score was excellent (ordinal α = .93).

Intervention

The intervention applied in this study was the Super Skills Multimedia program, a transdiagnostic cognitive-behavioral therapy protocol consisting of eight sessions, each approximately 40 minutes long. It is an adaptation of the in-person program with the same objectives, techniques, and content. Activities were revised and modified for multimedia presentation. The program is led by two digital characters who act as coping models and guide the learning process. Each session includes information on concepts children need to learn (such as the concept of anxiety or self-esteem) and activities that require children’s participation. For learning some techniques, such as relaxation or social skills training, sessions include videos with real models serving as references for participants’ practice. An eye mask appears on the screen as reinforcement for each activity the child completes. Access to the program is granted through a personalized password. Sessions occurred twice a week. After completing a session, participants were required to perform a task in their natural environment before the next session to help consolidate their learning. Information about the activities the child completed was accessible to therapists through a control panel, allowing them to monitor participants’ responses. Two therapists with training in the program were available via phone and email to address any issues or questions from parents. Parents also received a written guide with session objectives and assistance instructions.

Procedure

After obtaining permission from the respective ethics committee, information about the study was disseminated through social media and email contacts with school officials across Spain. The information included the study’s objectives, inclusion criteria, and a link to a website where the program’s details could be reviewed. Interested parents completed an online questionnaire, providing authorization for their children to participate in the study. Families who accepted participated by completing the assessment questionnaires. Parents were given instructions on how their children should complete the self-report measures. Each family was provided with a unique access code to the program, which is hosted on a website. Inclusion criteria for participation in the study were that children were between 8 and 12 years old, parents provided written informed consent, and the family had access to a device (computer or touchscreen) with internet access.

At the beginning of each week, parents received an email with instructions for completing the sessions, which were activated at a rate of two per week, with three days of rest between each session. Access to a session was only granted after the previous session and homework had been completed. After completing all eight sessions, parents and their children completed the assessment again. One year later, families were contacted again to complete the questionnaires once more.

Data Analysis

The data analysis was conducted using SPSS v26 software. Descriptive statistics were employed to examine the sociodemographic characteristics of the sample and gender differences among the children. Logistic regression analyses were carried out to identify differences in sociodemographic and outcome variables between the children who completed the intervention and those who dropped out of the study at the post-test or during the one-year follow-up.

Generalized Estimating Equations (GEE) were used to evaluate the effects of the intervention at the post-test and after one year of follow-up. Adjusted odds ratios (AOR) and 95% confidence intervals were obtained. The analyses were adjusted for participant’s age and gender, following an “intention-to-treat” approach, which includes data from all participants regardless of the number of sessions attended or assessments completed.

GEE is commonly applied to evaluate longitudinal data where repeated measurements are taken from a group of individuals. It allows controlling within-subject measurements to estimate changes over time through a variance/covariance matrix. This approach enhances the power of analyses with a large number of repeated measurements in small sample sizes, estimates the magnitude of variation in outcome variables over time, and permits the use of incomplete databases in follow-up measurements without excluding participants from the analyses (Liang & Zeger, 1986). Two pre-specified hypotheses were tested: (1) A contrast comparing scores between the pre-test and those obtained at the post-test, and (2) another contrast comparing scores between the pre-test and the one-year follow-up. The ordinal alpha of the scales used was calculated using R Studio 2022.02.2, due to their ordinal nature.

Results

Attrition

Figure 1 illustrates the participant flow during the study. The initial sample consisted of 283 children, which was reduced to 127 (44.87%) at posttest (those who completed the program and responded to the posttest). Only 33 (11.66%) participants responded to the follow-up one year later. Information regarding why families did not respond to the one-year follow-up is not available.

Figure 1. Modified CONSORT Flowchart for a Non-randomized, Single-arm Study of the Super Skills Multimedia Program.

External validity evidence was obtained, as there were no statistically significant differences in gender (p = .61), child’s age (p = .29), child-reported depression symptoms (p = .28), and anxiety and depression reported by the parents (p = .50 vs. p = .55) between the participants who responded at the post-test and those who dropped out without responding. However, differences were observed between these two groups in anxiety reported by the children, OR = .89, 95% CI [.83,.96], p = .005. Specifically, those who continued in the study had higher anxiety scores than those who dropped out and did not complete the post-test (M = 5.82, SD = 4.96 vs. M = 3.03, SD = 4.38), suggesting that children who were improving their symptoms tended to drop out of the study.

When comparing participants who responded to the one-year follow-up and those who did not, no statistically significant differences were observed in gender (p = .43), child’s age (p = .22), anxiety reported by the child (p = .85), and anxiety and depression reported by the parents (p = .91 vs. p = .45). However, there were differences between these two groups in self-reported depressive symptoms, OR = .89, 95% CI [.83,.96], p = .002. Specifically, children who continued in the study had higher scores for depressive symptoms than those who dropped out and did not respond to the follow-up assessment (M = 10.03, SD = 7.35 vs. M = 4.08, SD = 5.74). This result suggests that children with greater depressive symptoms tended to remain in the study longer than those who dropped out.

Intervention Effects

Table 2 displays the marginal means for anxiety and depression at pre-test, post-test, and one-year follow-up based on assessments reported both by the children and the parents. Table 3 provides the results of the generalized estimating equation models for the short-term and 12-month effects of the intervention. Figures 2 and 3 represent the marginal means of anxiety and depression symptoms, as assessed by the child and the parents, respectively.

Table 2. Marginal Means for Anxiety and Depression Symptoms at Pre-test, Post-test, and One-year Follow-up according to Children’s and Parents’ Assessments

Note. Higher scores indicate greater anxiety and depressive symptoms. CI = confidence interval.

Table 3. Estimates of 95% Confidence Intervals (CI) and Significance Tests based on Generalized Linear Models for the Intervention Effect on Outcome Variables

Note. CI = confidence interval; LL = lower limit; UL = upper limit.

Figure 2. Representation of Group Mean Scores in Anxious and Depressive Symptomatology at the Three Time-points, according to Children’s Self-report.

Figure 3. Representation of Group Mean Scores in Anxious and Depressive Symptomatology at the Three Timepoints, according to Parental Assessment.

Short-Term Efficacy

Compared to the pre-test, the program was found to be statistically effective in reducing anxiety levels as reported by children (p < .05) and parents (p < .001), as well as parental-reported depression levels (p < .001). Gender differences in depressive symptoms were observed at the post-test. Specifically, girls exhibited greater depressive symptoms than boys in assessments reported both by children (p < .01) and parents (p < .01).

Long-Term Efficacy

One year after the program’s implementation, assessments by the children indicated that those who received the intervention reduced their anxiety symptoms (p < .001) and showed a marginal reduction in depression symptoms compared to the pre-test (p = .07). According to parental assessments, children reduced their anxiety symptoms at the one-year follow-up (p < .001) compared to the pre-test. No statistically significant changes were observed in the variable of depression, according to parental assessments.

Discussion

This study aimed to assess the efficacy of a self-applied internet-based psychosocial program aimed at preventing emotional problems in Spanish school-aged children. While the intervention has evidence in its face-to-face version and as an indicated prevention through self-application, its utility as a preventive program for promoting socio-emotional skills in a community population, as well as its short-term and long-term effects, was unknown until now.

The results indicate that Super Skills Multimedia successfully reduced anxiety scores according to assessments completed by parents and in children’s self-reports. Regarding the examined symptoms of depression, the parents reported children’s lower scores after completing the intervention. Previous studies focused on the same program showed some discrepancies between informants on the results. Parents’ assessment showed improvements in some variables, whereas only the children’s assessment reported improvements (Orgilés, Morales, et al., 2023). Informant discrepancies in assessing youth mental health problems seem to be common and should be considered (see De Los Reyes et al., 2022).

The intervention is based on a transdiagnostic approach, which means that it targets variables acting as risk factors for different emotional problems and disorders. To this end, it includes various components that, to varying degrees, address protective socio-emotional competencies: social skills, cognitive therapy, emotional education, emotional regulation, problem-solving, exposure, and behavioral activation, among others. Although it addresses all these competencies transversally, the results show that these components have a greater impact on factors affecting anxiety than on those affecting depression. This is consistent with previous findings on the program (Essau et al., 2014), where it has been observed that the positive impact of the Super Skills program on symptoms of anxiety is consistent with those reported in the original study. This was particularly expected given that the Super Skills program has a strong component of social skills training and other techniques to promote anxiety regulation.

As expected, we observed that the internet-based program effectively reduced emotional symptoms, expanding the evidence of the effectiveness of the face-to-face program. As the components included in both versions remain the same, the active principles of the two modalities are the same, achieving similar improvements in socio-emotional well-being for participants in the online version as in the face-to-face version, with lower scores in the post-test evaluations. However, Super Skills Multimedia achieved better results in reducing anxiety scores compared to depression, and this improvement was only reported by parents and not by the children themselves. Although the explanation is not clear, the children may have had more difficulty completing the self-reported questionnaire for depressive symptoms or, despite including components such as behavioral activation, the program is more focused on anxious symptoms than depressive symptoms.

In the long term, the program seems to maintain the results achieved in the short term. Children reported lower anxiety scores and, with marginally significant results, lower depression scores compared to the pre-test. Similarly, according to parental assessments, children reduced their anxiety symptoms at the one-year follow-up compared to the pre-test. However, there were no significant changes in depression scores according to parental assessments. The long-term maintenance of effects is also observed in the face-to-face version of the program, with even more positive results in the long term than in the short term, both in the current self-applied version and in the face-to-face intervention. As the program is a transdiagnostic protocol for learning socio-emotional skills, it is not only aimed at addressing the child’s current problems but also involves the acquisition of psychological resources that can be applied in future life situations. The application of these skills in daily life facilitates an increase in socio-emotional competence to cope with situations that cause distress, ultimately leading to improved emotional well-being over time.

In general, participants improve in the long term, but it seems that the short term improvements are seen earlier in boys. There are several tentative hypotheses: Boys may be less likely to reflect on negative emotional experiences, which could lead to an apparently faster improvement in symptoms in the short term. Also, girls are often socialized to be more expressive and aware of their emotions, which could make them more sensitive to their emotional states and therefore slower to show observable improvement. In the long term, this greater awareness and processing may result in a more sustainable resolution of emotional problems. Lastly, cultural and family expectations about gender roles can also influence how boys and girls respond to treatment.

Of the participants assigned to the intervention, more than half did not complete the assessment tests, dropping out of the program without finishing all the sessions. The dropout rate was even higher for the long-term follow-up, with more than 80% of participants not completing the long-term assessment. An important point to consider in understanding the high dropout rate is that, according to the “intention-to-treat” analyses, children who dropped out of the program had lower anxiety scores than those who completed the intervention. This suggests that those who were improving tended to abandon the program. A similar pattern was observed for long-term effects; children who completed the long-term assessment had more depressive symptoms than those who dropped out of the study. This indicates that those who had more symptoms tended to continue and complete the program and report short- and long-term results, as observed in previous studies (March et al., 2021).

The study has certain limitations to consider. Firstly, the sample size was considerably reduced, especially for the long-term follow-up. Telephone calls and email reminders were made to parents to encourage completion of the long-term assessments, but the desired number of responses was not achieved. It was more common for children with fewer symptoms to drop out of the study. This suggests that those who remained in this study were those with more severe symptoms of anxiety and depression. Possibly, those with mild levels of depressive and anxious symptoms improved shortly after starting treatment. The perceived improvement may have led them to decide to abandon both the treatment and the study. This must be considered as a limitation since drop out rates may have influenced the results and might have impacted the generalizability of the findings. However, high dropout rates in longitudinal studies are common (Garfield, 1994; Wierzbicki & Pekarik, 1993). It is estimated that between 35-75% of pediatric patients leave mental health services before the health provider deems it appropriate (Kazdin, 1990; Miller et al., 2008). This suggests that children and adolescents often receive less than the adequate dose to address their issues. A high dropout rate from anxiety treatments results in inefficient use of resources and can generate additional costs since, in the future, these children with anxiety symptoms tend to return to the health system (Armbruster & Kazdin, 1994; Kazdin, 1990). Despite the methodological limitations of high participant dropout rates, longitudinal studies are considered valuable and necessary (Gustavson et al., 2012). The reasons why dropout rates tend to be high in longitudinal studies are diverse and vary depending on the sample. Knowing these reasons would improve treatment adherence and retention rates in longitudinal research. However, this information is often not provided in studies (Kazdin, 1990). Among the most common reasons are the lack of motivation or ongoing interest from participants and their parents, the prolonged duration of the study, and logistical difficulties such as scheduling conflicts (see Kazdin, 1990; Kendall & Sugarman, 1997). Additionally, this version of the program required the support of parents as co-therapists, which apparently represented additional work for the parents, and not all were available. The use of multimedia content requires that families have at least one electronic device and have basic computer knowledge. Another limitation is the lack of a control group, which was not included in this study to avoid ethical issues related to leaving a group of children without intervention for 12 months. Active control groups should allow comparisons with other treatments, or the standard treatment. This approach not only should addresses eventual ethical concerns but also enhance the validity and reliability of the outcomes. Future studies to evaluate the intervention should consider incorporating this kind of study design.

Despite the limitations mentioned, the study has several strengths. Firstly, it provides empirical evidence for the only self-applied program available in Spain for intervening in childhood emotional problems. The protocol is based on a program with accumulated evidence, but it leverages the advantages of technology to facilitate access for children who may not otherwise be able to attend in person. Furthermore, the long-term efficacy confirms that acquiring socio-emotional skills can be a valuable strategy for improving well-being and preventing future psychological problems. In fact, some studies have highlighted the importance of designing preventive interventions based on developing socio-emotional skills to combat bullying (Espino et al., 2023). In this regard, the face-to-face application of the Super Skills program has proven effective in improving social relationships in children, reducing peer problems, and increasing prosocial behaviors (Melero, Orgilés, et al., 2021). Interventions that enhance socio-emotional skills have also been recognized as crucial for suicide prevention; for example, the guide “Vivir la vida [Live Life]” (Organización Mundial de la Salud [World Heath Organization], 2021) recommends promoting socio-emotional skills for suicide prevention in adolescents. The program achieves improvements in child well-being in just eight sessions, but it can also be a valuable intervention for preventing problems such as bullying or suicidal behavior, as well as preventing the development of other disorders in adolescence or adulthood. It is a highly protocolized intervention, which facilitates its self-application both at home and in a school setting, and it is well-received by children due to the advantages that technology offers for delivering content. As a future research direction, it would be interesting to explore the program’s effectiveness in preventing issues where emotion-regulation problems play a role, such as situations of rejection, bullying, or self-harming behaviors.

In conclusion, this study provides empirical evidence of a tool for developing socio-emotional competencies and preventing emotional problems in schoolchildren. The practical implications of this study are clear: Super Skills Multimedia is a useful, effective psychosocial resource that can make psychological care more accessible to the child population.