Engagement with Activity Monitoring During a Behavioral Activation Intervention: A Randomized Test of Monitoring Format and Qualitative Evaluation of Participant Experiences

Abstract

Behavioural activation (BA) is an efficacious treatment approach. Activity monitoring is a key component of brief BA treatments; however, no studies have examined the most efficacious format for monitoring. The present pilot study tested brief versus intensive activity monitoring approaches during a BA intervention administered in a college orientation course. Outcomes characterised (1) engagement with the treatment protocol via activity monitoring and (2) participant qualitative experiences with monitoring and the intervention as reported during focus group interviews. Four course sections were randomly assigned to receive monitoring forms that were brief (assessed activities three times daily) or intensive (assessed activities hourly). Forms were provided electronically to students via a web-based platform which tracked completion. There were no significant differences in monitoring frequency (38.0 vs. 23.0 days; p = .154) or the duration of monitoring engagement (62.0 vs. 36.0 days; p = .054) between the brief and intensive conditions. Qualitative findings suggested that participants in both conditions found utility in activity monitoring, particularly during the first month as they transitioned to college. Overall, findings indicated that participants may find utility in monitoring during the first month of a BA intervention using either brief or intensive monitoring forms.

Clinical Trials Registration Number: NCT04038190

Introduction

Behavioural activation (BA) is a brief treatment approach designed to help individuals increase their engagement with reinforcing activities in their environment. BA is based on the premise that a deficit in reinforcement from the environment may yield difficulties with psychosocial functioning. For example, a decrease in reinforcement from the environment has been hypothesised to lead to dysphoric mood and decreased engagement with activities, which may lead to depression (Ferster, 1973; Lewinsohn, 1974; Ramnerö, Folke, & Kanter, 2016). A deficit in alternative sources of reinforcement from the environment may also yield engagement in risky substance use as a primary source of reinforcement (Murphy, Correia, & Barnett, 2007). Thus, BA targets the reinforcement system and seeks to increase engagement with sources of reinforcement to improve functioning. BA was originally developed to treat depression and its efficacy has been well established (Ekers et al., 2014). More recently, BA has been used to target risky substance use, and initial evidence indicates its efficacy for a variety of treatment settings (Fazzino, Bjorlie, & Lejuez, 2019). However, despite its efficacy for both depression and risky substance use, very limited research has focused on factors that may promote engagement with a BA intervention, which may be important for understanding potential contributors to treatment efficacy.

A key component of BA is homework, which involves monitoring of engagement with reinforcing activities in an individual's environment and, depending on the treatment target, monitoring of mood and/or substance use. Activity monitoring facilitates individuals in aligning their activities with their values (as identified during treatment), thereby implementing BA concepts into their daily lives. Monitoring can involve tracking of daily activities, as well as planned activities that may be assigned by a therapist during treatment, or activities that are patient-initiated and scheduled. Despite the core function of activity monitoring in treatment, limited research has measured activity monitoring engagement and completion during BA. Preliminary studies that examined monitoring engagement or completion were small, pilot investigations of BA for depression (Busch, Uebelacker, Kalibatseva, & Miller, 2010; Crits-Christoph et al., 2021; Folke et al., 2015; Funderburk, Pigeon, Shepardson, & Maisto, 2020; Ryba, Lejuez, & Hopko, 2014). Across the pilot studies, monitoring engagement and compliance varied substantially (36–84%). For example, two pilot studies of BA in primary care reported that participants completed <50% of monitoring reports during the treatment period [36% in a 2-session treatment protocol (Funderburk et al., 2020); 47% in a 10-session protocol (Busch et al., 2010)]. In a two pilot studies of BA for depression in outpatient mental health treatment settings, homework consisted of monitoring activities and mood daily, as well as completing periodic workbook materials (Crits-Christoph et al., 2021; Folke et al., 2015). Overall completion of homework was high across the 6-week protocol (84%) (Crits-Christoph et al., 2021) and moderate over the 12-week treatment protocol (69%) (Folke et al., 2015). Finally, one BA study among breast cancer survivors (N = 23) reported that the completion of activities was very high and remained stable throughout the 6 weeks of treatment (Ryba et al., 2014). Overall, initial evidence indicates that homework engagement during BA treatment may be variable across studies.

Despite initial evidence suggesting high variability in monitoring engagement across BA studies, it is largely unknown what factors may contribute to the observed variability in engagement. Daily monitoring forms are typically structured to be intensive in format and collect hourly activity reports each day (Lejuez, Hopko, Acierno, Daughters, & Pagoto, 2011). Intensive daily monitoring forms may facilitate more frequent monitoring of activities and may contribute to greater treatment efficacy; however, hourly reporting may also yield a higher response burden for participants and may result in early discontinuation of monitoring among some participants and samples. Thus, the intensity of the monitoring forms could contribute to variability in monitoring compliance observed in the prior studies, most of which used the intensive monitoring format. Daily monitoring forms could be modified to employ a briefer format. While a briefer format would collect less fine-grained information on engagement with activities each day, form brevity and lower participant burden may facilitate greater engagement with monitoring over time. However, no prior work has directly examined whether the format of activity monitoring forms may impact engagement with daily monitoring during a BA intervention.

A third limitation in the literature is that few studies have inquired about individuals’ experiences with key components of BA treatment, including activity monitoring, which may yield insights into factors that promote engagement in treatment. Most qualitative research has been conducted in pilot studies of BA for depression or substance use, during which participants were interviewed in focus groups that inquired about their overall experiences in treatment. Although not a primary focus of the groups, some information emerged regarding participant experiences with monitoring. For example, among three studies of BA for depression, all reported that participants identified utility in activity monitoring (Choi, Caamano, Vences, Marti, & Kunik, 2020; Collado, Castillo, Maero, Lejuez, & MacPherson, 2014; O'Neill et al., 2019). For example, in a pilot study of culturally adapted BA treatment for Latino participants with depression, intervention participants reported that the daily monitoring of activities was easy and uncomplicated (Collado et al., 2014). In a BA intervention adapted for group-based depression treatment, participants indicated that daily monitoring facilitated their sustained focus on the programme and related activities (O'Neill et al., 2019). Finally, in a study that developed a BA intervention for young adults who smoked cigarettes, participants reported that daily monitoring of activity engagement and cigarette use ahead of a quit attempt was helpful (MacPherson, Collado, Ninnemann, & Hoffman, 2017). One participant expressed concern about the amount of homework required; however, further details regarding the participant's concern were not provided (MacPherson et al., 2017). Thus overall, there has been limited qualitative research conducted with participants in BA treatment, which is needed to understand factors that may influence treatment engagement.

In summary, there are gaps in the literature regarding factors that may facilitate or hinder engagement with BA treatment via activity monitoring, and limited understanding of participants’ qualitative experiences with key components of a BA intervention, all of which are important for understanding treatment engagement. In psychotherapy treatment, engagement with treatment is a key factor that influences the dose of treatment received and overall treatment efficacy (Callan et al., 2019; Glenn et al., 2013; Spielmans & Flückiger, 2018). With regard to homework and activity monitoring specifically, engagement with monitoring may have a substantial influence on the degree to which individuals apply what they learn during treatment sessions to their daily lives, which may also influence treatment efficacy (Kazantzis, Whittington, & Dattilio, 2010, Kazantzis et al., 2016). Thus, examining key contributors to BA engagement may be informative for enhancing participation during treatment and possibly treatment efficacy. Thus, the current pilot study was designed to test whether the format of daily monitoring forms may influence monitoring engagement during an intervention. Specifically, the study examined whether a brief monitoring form that measured activities at three times per day (morning, afternoon, and evening) yielded greater daily monitoring engagement than the standard, intensive (hourly) monitoring form during a BA intervention. In addition, the study conducted qualitative focus group interviews to understand participant experiences with activity monitoring in the brief versus intensive monitoring conditions (primary focus, with the intent of comparing across conditions), and participant experiences with the BA intervention overall (secondary focus). The study administered a BA intervention in a semester-long freshman orientation course for undergraduate college freshmen; four course sections were cluster-randomised to use either brief (n = 2 course sections) or intensive (n = 2 course sections) daily monitoring forms. Qualitative feedback from participants was collected during focus groups conducted at the end of the semester. We hypothesised that due to their brevity and lower burden on participants, brief monitoring forms would yield greater engagement in daily monitoring compared to intensive monitoring forms. Engagement with the treatment via daily monitoring was measured as (1) the frequency of monitoring during the intervention, to characterise total monitoring reports completed, and (2) the duration of engagement with monitoring, to characterise the extent to which participants remained engaged with monitoring during the semester. Participant experiences with daily monitoring in the brief versus intensive conditions (primary outcome) and with the BA intervention overall (secondary outcome) were characterised using qualitative interview data from focus groups conducted at the end of the semester.

Methods

Study Overview

The study was approved by the University's Institutional Review Board. The study was conducted in a University 101 Freshman Orientation Course (UNIV 101), a semester-long freshman orientation seminar that introduces college freshmen to academics and the campus environment. The BA intervention content was integrated within the UNIV 101 course and administered in the first 5 weeks (25 days) of the course. Study staff ran four sections of the UNIV 101 course during the Fall 2019 semester. Before the semester began, the course sections were randomised to either the intensive (n = 2 course sections) or brief (n = 2 course sections) daily monitoring conditions using a computerised random number generator and an assignment procedure. Freshmen signed up for course sections via the university registrar. Each course section had between 10 and 19 students enrolled.

During the second week of classes, students were invited to participate in the study during a class period. The consent was explained in detail to the students and students were notified that their participation was completely voluntary, and that they could withdraw at any time. Students were also notified that their choice whether to participate in the study would not influence their grade in the course. Interested students provided full written informed consent to participate. Participants filled out surveys at three times during the fall semester (August, October, and December) and participated in a focus group at the end of the semester. Students received extra credit in the UNIV 101 course for their participation.

Quantitative Measures

Monitoring forms (engagement with treatment)

Monitoring forms were provided to students via a web-based platform and could be accessed from any computer or mobile device with internet access. Participants were provided individualised links to the monitoring platform, which obviated the need to log in and was designed to streamline form access and completion. Participants were automatically presented with the type of monitoring form (brief or intensive) that corresponded to their UNIV 101 course section's randomisation assignment. Participants were introduced to activity monitoring during the second week of class. Instructors explained the rationale for daily monitoring to participants, which was to facilitate their application of material learned in class to their daily lives. Participants were then introduced to the daily monitoring forms and their components, which included text boxes to describe activities that corresponded with each time block and rating scales that corresponded with the activities. Participants were instructed to rate each activity regarding (1) how enjoyable it was on a scale of 0–10, and (2) how important it was to their values, on a scale of 0–10. In addition to the activity monitoring and ratings, participants rated their overall stress for the day, on a 0–10 scale. Participants were instructed to complete the monitoring forms daily, in accordance with the form structure (3× daily or hourly). Instructors helped students address issues regarding technical problems as they arose. Data on the frequency in which participants completed daily monitoring reports and the date of their last monitoring report submission were used in analyses as outcome measures of engagement with the treatment.

Demographic information

Participants completed an electronic survey that inquired about the following demographic information: age, sex, race, ethnicity, year in school, and location of residence (on or off-campus).

Qualitative Measures

Focus group interviews (participant experiences with monitoring and the intervention)

All focus group participants consented to observation, audio recording, and transcription of the group conversation. Focus groups were conducted at the end of the semester. Focus groups were conducted separately for individuals from the brief and intensive monitoring courses. Focus groups were held in a neutral location on campus (i.e., a conference room). In order to follow best practices for focus groups (Manning & Kunkel, 2014), students were offered an incentive to participate in the form of snacks. Consistent with Tracy (2013), a primary moderator ran the groups and a secondary moderator observed participants and recorded notes about verbal and nonverbal behaviours that may have been missed by the recording. The moderators were not affiliated with the UNIV 101 course, nor other data collection-related activities. Focus groups were expected to contain five to ten participants (Lindlof & Taylor, 2017); however, some of the focus groups in this study were smaller due to the challenges associated with conducting the groups at the end of the semester when students have limited time and competing demands for academic work (Manning & Kunkel, 2014; Tracy, 2013).

Participants in the brief and intensive monitoring focus groups were asked the same questions for the purposes of comparison and contrast. The focus group interview guide had two sections with separate questions for each one: (1) the activity monitoring forms and (2) the role of the course in helping students adjust to life as new college students. Sample questions about the activity monitoring forms included questions like: ‘What are your thoughts about the daily monitoring forms, wherein you tracked your daily activities and how you spent your time on a given day?’ and ‘What worked well on the daily monitoring forms? Did they help you schedule your day?’ Sample questions about the role of the class in adjusting to college life included questions like: ‘How did taking this class impact your adjustment to college, and can you give any specific examples?’ A full list of the focus group questions appear in the supplemental information section and were the basis for outcome measures of participant experiences.

A total of four focus groups were conducted, three of which were conducted with participants in the intensive monitoring condition and one with participants in the brief monitoring condition. A total of n = 10 participants provided feedback across the three intensive monitoring condition focus groups and n = 3 participants provided feedback in the brief monitoring condition focus group.

Data Analysis

Quantitative analysis: engagement with activity monitoring

To test whether brief monitoring forms garnered greater engagement with daily monitoring compared to intensive monitoring forms, both descriptive statistics and Mann–Whitney tests were used. In line with directional hypotheses, one-sided tests were conducted. Engagement was examined as: (1) the total number of daily monitoring reports completed during the semester and (2) the duration of engagement with daily monitoring forms, defined as the last day a monitoring form was submitted during the semester. A post hoc power calculation indicated that based on our sample of N = 59 participants, we had adequate statistical power to detect effect sizes equivalent to a Cohen's d = .67 or greater, which is considered moderate to large.

Qualitative analysis: participant experiences

Overall, across the four focus groups, 161 min of audio were recorded, with an average focus group length of 40 min (range: 15–70 min). Given the complex nature of focus group recordings, a professional transcription service was used to transcribe all focus group recordings. Upon receipt of the focus group transcriptions, the primary group moderator listened to the audio files while reading each transcript for quality control before analysis. Specifically, the quality control review consisted of: (1) cleaning transcript files (e.g., removing typos); (2) ensuring that the correct number identifier was assigned to each focus group participant; and (3) verifying that data were de-identified. In total, the transcriptions yielded 121 single-spaced pages of text for analysis. In addition to the transcribed text, a total of 11 pages of single-spaced pages of text notes were written by the second moderator during the focus groups, which were also used for analysis.

Given that the research area of the present study is relatively bereft of qualitative research methods, the initial coding of transcripts was an iterative process (Bhattacharya, 2017; Tracy, 2013) that included computer-assisted QSR NVivo qualitative analysis software and the comments function in Microsoft Word (Lindlof & Taylor, 2017). Open and axial coding techniques (Manning & Kunkel, 2014; Miles & Huberman, 1994) were used to examine the data for prominent themes and/or patterns. Transcripts were compared and contrasted for themes using constant comparison techniques (Charmaz, 2006; Thornberg & Charmaz, 2012) in order to develop more elaborate codes and themes.

Both group moderators (primary and secondary) read the transcripts multiple times in a line-by-line fashion. The two moderators coded independently to start, and then the two coders discussed their thoughts regarding initial themes. Data were subsequently recoded using the new information gleaned from the discussion and the moderators arrived at a list of final themes. Given the focus of the present study on BA content and daily monitoring, the findings are reported for topics 1 (daily monitoring) and 2 (role of the course in adjustment to college).

Results

Participant demographic characteristics by the condition are presented in Table 1 and summarised herein. A total of N = 59 students from the four course sections consented to participate, which represented 72% of students enrolled in the course sections. Of the participants, 98% were 18 or 19 years old; approximately half were female (54.2%; 32/59), and most identified as White, non-Hispanic (71%; 42/59), or White, Hispanic (8.5%; 5/59). Chi-squared tests indicated no significant differences between intensive and brief monitoring conditions on age, sex, race, or ethnicity (Table 1).

Table 1 Demographic Characteristics of Participants (N = 59)

Quantitative Results: Engagement with Activity Monitoring

Descriptive statistics for the frequency and duration of engagement with activity monitoring are presented in Table 2 and depicted in Figures 1 and 2. Figure 1 presents the frequency of total daily monitoring reports completed across participants and Figure 2 presents the total duration of time (in days) over which participants completed monitoring reports. In the comparison of the brief and intensive monitoring conditions, we tested potential differences in monitoring the frequency and duration of engagement. There were no significant differences in monitoring the frequency (23.0 vs. 38.0 days; U = 341.0; p = .154; r = .186) or duration of engagement (36.0 vs. 62.0 days; U = 285.5; p = .054; r = .255) between the brief and intensive monitoring conditions as indicated by Mann–Whitney tests (Table 2).

Figure 1. Total daily monitoring reports completed by the participant.

Figure 2. Duration of daily monitoring by the participant.

Table 2 Descriptive Statistics for Daily Monitoring Frequency and Engagement

Note. IQR = interquartile range; BA = behavioural activation.

Qualitative Results: Participant Experiences with Monitoring and the Intervention

Specific to our qualitative analyses, findings are presented from the two topical areas addressed during the interviews: (1) experiences with activity monitoring and (2) the role of the class in facilitating students’ adjustment to college. Findings from each topic are presented from the brief monitoring condition first, followed by the intensive monitoring condition, for direct comparison across conditions by topic.

Topic 1: activity monitoring

Brief monitoring condition

Theme: highlighted habits and patterns. Students in the brief monitoring condition reported that for approximately the first month, the activity monitoring forms helped them improve their time management, organisation, and allowed them to see a routine or patterns in behaviour. For example, one student reported that because the monitoring form ‘needed to be pretty detailed,’ the monitoring process helped her ‘really see a pattern [with] my punctuality, my attendance, and stuff like that’ (P2, Group 8). Students highlighted the benefit of starting the daily monitoring at the beginning of their first semester of college, as they were establishing their daily habits. Students specifically noted that the monitoring process helped them identify ways they could improve their time management. For example, one student mentioned ‘it really helped me see like … how much time I had off to do other things … And it helped me like, see like, oh, I have this amount of time where I'm not doing anything. Maybe I should add some things … Like, the organizing part? It's been helping me with that’ (P1, Group 8). In contrast, another student mentioned that the monitoring helped her identify that her schedule was too full: ‘I was kind of overbooking myself when it first started … So I had to back away from some things’ (P2, Group 8). Finally, one student highlighted the utility of learning organisational skills through daily monitoring, which benefited her later in the semester: ‘It helped me become more organized as a person and just like, carry those, like traits throughout the semester’ (P3, Group 8).

Theme: lack of incentive to continue monitoring over time. Students in the brief activity monitoring condition reported decreased motivation to complete the monitoring over the semester because they were not incentivized with points toward their grade to complete the monitoring. One student reported, ‘just knowing that you're not being graded on it. You're just like eh, I don't want to do it’ (P1, Group 8). The students felt that they would have engaged with monitoring more if they were incentivized to do so. For example, one student said, ‘When you go about telling brand new college students something that it's not, like that it's for points, it's like optional … So, they're probably not going to do that. They prioritize it very low … in their list of things to do’ (P3, Group 8). Students in the brief monitoring condition also recommended providing points for completing the monitoring ‘I know they made it like, for points, people would definitely do it’ (P3, Group 8).

Theme: experiences with the web-based system and tracking preferences. All participants in the brief daily monitoring condition reported frustrations with the user interface of the activity monitoring forms. Specifically, students indicated that the forms required multiple clicks to get to the monitoring form and could have been better designed to facilitate the user experience. For example, one student indicated challenges with navigation despite the brevity of the monitoring forms: ‘I mean it only took like, what, like five minutes …  but it felt like it could've been done in a little easier way’ (P3, Group 8). Two students indicated a preference for other modes of monitoring. Specifically, both students indicated that they preferred to track their activities and time in their personal calendars or daily planners. They also suggested that tracking on paper may have been preferable.

Intensive monitoring condition

Theme: highlighted habits and patterns. Students in the intensive monitoring condition noted that the activity monitoring process helped them identify trends in how they spent their time. More specifically, students identified that the monitoring forms helped them notice patterns in their daily routines and organise their time. Students also reported that the monitoring forms facilitated comparative observations of their time spent on academics versus other areas, such as social and recreational activities. For example, one participant mentioned ‘I found out I was doing more being with my friends than studying’ (P1, Group 7). Overall, participants indicated that early in the semester, the daily monitoring helped them organise their time more effectively, in line with their priorities and goals.

Theme: convenience of web-based platforms despite some technical problems. Students in the intensive monitoring condition appreciated the online format of the monitoring process. Students indicated that the online monitoring process was efficient and that it was easy to access and complete the forms. Furthermore, three students reflected that it made the monitoring process more efficient than it may have been using other formats. For instance, one student reflected: ‘Doing it online is pretty helpful …  and I'm sure it's a lot more efficient than doing it on paper’ (P1, Group 1).

Despite overall appreciation of the online format, students expressed some frustration with intermittent technical difficulties with the online platform or usability features. Specifically, some students indicated that they had intermittent difficulty in opening or accessing the monitoring forms if they had unstable Wi-Fi connections. One participant mentioned they ‘just couldn't get it to open every now and then. Sometimes it was a little buggy with that.” Several students reported an instance in which they forgot to save their data before exiting the forms, which occurred when they were rushing to complete the form.

Theme: decreased utility of monitoring over time and lack of incentive. Despite the initial benefit of activity monitoring in facilitating time management, students overall indicated that as the semester progressed, the monitoring became less useful. Specifically, students indicated that over time, they felt as though they were not learning new information about themselves or their habits, which led to the monitoring forms seeming tedious. Several students referred to the monitoring as ‘busy work ’ and a ‘hassle ’ over time. Furthermore, students reported decreased motivation to complete the forms over time because completing of the monitoring was not incentivized; they did not receive points toward their grade for completing the daily monitoring. Furthermore, the students reported that such a large amount of effort to complete the intensive daily monitoring forms was not adequately compensated. For instance, one student reported: ‘I was like, “whoa, there has to be an incentive if I'm gonna do it”’ (P4, Group 4). Several students recommended providing an incentive (e.g., extra credit) to increase monitoring compliance.

Topic 2: role of class in adjustment to college

Brief monitoring condition

Theme: coping skills. Students in the brief monitoring condition valued how the course helped them to cope by instilling awareness of themselves and areas in which they could improve. Some students stressed the importance of recognising how to achieve both independence and balance between college and home life. One student emphasised how the course focused on ‘taking care of yourself, but also taking care of responsibilities, academic and like self-wise. So I think that kind of touched on independence, you know. You have to do that, you have to take care of yourself. You have to take care of your class work’ (P2, Group 8). Other students reflected that class helped them develop self-awareness and appreciation of their strengths, while still encouraging growth where needed. For instance, one student described ‘we often don't appreciate our strengths and who we are … So it's sometimes good to reflect on that, which we did’ (P3, Group 8).

Theme: connecting with students and the university community. Students in the brief monitoring condition appreciated that they could relate to and connect with other students in their class. Specifically, there were positive reports of interactions with other students and their comments. One student expressed ‘it was interesting to like hear what other students had to say and like what they did in the week’ (P1, Group 8). The students also highlighted the importance of developing connections with faculty and other campus members. As one student described: ‘We kinda learned like, not networking skills, but…how to really connect with people on campus and different people like, authorities … it helped me to be able to talk to different teachers or different people around campus … and get to really know them and build a relationship with them ’ (P2, Group 8).

Intensive monitoring condition

Theme: coping skills. Students in the intensive monitoring condition appreciated that the course offered assistance in transitioning to college life. Specifically, students indicated that the course helped them improve their time management skills, achieve balance between academics and social life, and develop skills to cope with stress. For example, students reported that the class helped them adjust to the college setting through discussions of what to expect in college, challenges to anticipate, and coping strategies to use during the transition. One student described how the class helped: ‘it's given me insights on how to … deal with the transitions, and how to manage my time, and the classes, and balance everything’ (P1, Group 4). Some students reported that the course helped them develop new coping skills that they needed during the transition to college. As described by one student: ‘what I found useful about the class is that it taught me things that I did not know, like certain coping mechanisms that we learned about and talked about in class. They sort of helped me with my transition to KU, and helped me manage my classes from the beginning of the semester to now … and how to manage your stress, how to balance out what's important, what's enjoyable, what's not important.’ (P1, Group 1)

Theme: connecting with students and campus resources for support. The participants in the intensive monitoring condition appreciated the opportunity to connect with other students and learn about campus resources. Throughout the course, students connected with each other and established supportive relationships. One student described ‘not only does it help you, but you are helping the other person…which is nice’ (P3, Group 7). Students also emphasised that they learned about resources on campus and life skills that they had not previously learned about. As described by one student, ‘all of that, it's pretty common sense stuff but it's important cause they don't teach it in high school’ (P3, Group 7).

Brief and intensive monitoring conditions: comparison and contrast

Overall, participants from both brief and intensive monitoring conditions reported that, initially, the activity monitoring forms helped them identify patterns and improve their time management as they started college. However, participants from both conditions reported that over time, the monitoring became tedious and was less useful. Students in both conditions expressed a desire to receive incentives (e.g., points toward their grade) to complete the daily monitoring. However, notably, students in the intensive monitoring condition emphasised the discordance between the magnitude of time required to complete the monitoring and the lack of incentive, whereas the students in the brief monitoring described monitoring as a decreasing priority during the semester in the absence of benefit to their course grade. Both groups reported that the BA course helped them identify coping strategies and implement them into their daily lives, and appreciated connecting with other first-year students and campus resources for support.

Discussion

BA is an efficacious treatment approach for psychosocial problems that involve the reinforcement system, particularly depression and risky substance use. Despite widespread use of the intervention, no prior studies have examined the factors that may influence engagement with core components of BA treatment such as the use of brief versus intensive activity monitoring forms. The present study was designed to test whether the format of activity monitoring forms, a core component of BA treatment, influenced engagement with daily monitoring during a BA intervention for college freshmen. The pilot cluster-randomised trial assigned four course sections to receive either brief (3× daily) or intensive (hourly) activity monitoring forms. The study also conducted qualitative interviews to understand participants’ experiences with the brief or intensive monitoring forms (primary focus), as well as their experiences with the BA intervention overall (secondary focus). Results indicated that there were no significant differences in the frequency or duration of daily monitoring engagement between the brief and intensive monitoring conditions. Findings from the qualitative interviews indicated that participants in both the brief and intensive monitoring groups found utility in daily monitoring in the first month; however, the monitoring became less useful over time. No substantive differences in the experiences of participants with the brief versus intensive monitoring forms emerged from the qualitative interviews. Overall, findings indicated that participants may find particular utility in monitoring during the first month of a BA intervention using either brief or intensive monitoring forms.

The present study was the first to test whether differences in the format of activity monitoring forms may influence participant engagement with daily monitoring during a BA intervention. Overall, there were no significant differences in the frequency or duration of monitoring for participants in the brief versus intensive monitoring conditions, suggesting that either format may be useful for garnering monitoring compliance during a BA intervention for college freshmen. However, given that the study was a pilot study and was not powered to detect smaller effects, the nominal differences observed across groups may be worth considering, as they may have yielded some clinical utility in the context of the intervention. For example, although not significantly different (p = .054), participants in the brief monitoring condition remained engaged with monitoring for almost double the length of time compared to participants in the intensive condition (median for brief = 62 days versus median for intensive = 36 days). The longer duration of monitoring over time may have been clinically useful by facilitating participants in applying the BA concepts in their daily lives throughout the semester, and notably, beyond the delivery of the main intervention content (in the first 25 days). Similarly, participants in the brief monitoring condition completed a median of 15 additional monitoring reports compared to participants in the intensive monitoring condition (brief: 38 vs. intensive: 23), a difference that yielded approximately 2 weeks of additional monitoring compared to the intensive monitoring participants. Overall, our findings indicate that there were no significant differences in monitoring engagement across the groups; however, using brief monitoring forms that assess activities across three times of day may be useful for nominally extending engagement with daily monitoring during a BA intervention for college freshmen. However, given the small sample (N = 59) for this pilot trial and the limited statistical power, future work with larger samples should seek to replicate our findings.

Our qualitative findings provided further insights regarding the experiences of participants in both the brief and intensive monitoring conditions; however, the experiences of participants across both conditions largely aligned despite the differences in the format of their activity monitoring forms. Overall, participants from both monitoring conditions reported that particularly in the first month, activity monitoring helped them understand how they were allocating their time and facilitated planning activities in alignment with their goals and priorities. Thus, findings indicate that using the monitoring forms may have aided students in implementing the BA concepts introduced in class into their daily lives. The findings align with prior studies in the literature conducted among clinical samples, in which participants reported that daily monitoring helped them sustain a focus on planning and completing activities that aligned with their goals and values throughout treatment (Choi et al., 2020; O'Neill et al., 2019). However, after the first month, participants reported that the monitoring became tedious and less useful. Students across both conditions recommended providing incentives for completing the monitoring with points toward their course grade. Notably, students in the intensive monitoring condition emphasised the discordance between the magnitude of time required to complete the monitoring and the lack of incentive, whereas the students in the brief monitoring condition described monitoring as a decreasing priority during the semester in the absence of benefit to their course grade. The mention of incentives was the only time during the focus group interviews in which participants appeared to differentially emphasise the time required to complete the brief versus intensive monitoring forms. Overall, qualitative feedback largely indicated that participants found utility in activity monitoring in both the brief and intensive monitoring conditions in the first month of the intervention, and that the utility of monitoring declined thereafter.

Our study also contributes to a small body of literature that examined participant experiences with a BA intervention qualitatively. Overall, participants indicated that the BA course was useful during their adjustment to college because it facilitated students in organising their time and aligning their activities with their priorities, thereby facilitating balance between academics and other life areas (e.g., social). Students also reported that the course addressed important coping skills and helped students connect with others for support. Our findings are consistent with those of other qualitative studies of BA conducted in clinical samples, in which participants emphasised how BA helped them organise and structure their time and activities in a meaningful way (Choi et al., 2020; Collado et al., 2014; O'Neill et al., 2019) and facilitated greater social support and connectedness (Choi et al., 2020; MacPherson et al., 2017; O'Neill et al., 2019). Thus overall, participants in both conditions highlighted the utility of the BA content in facilitating their adjustment to college.

Findings from this pilot study of BA administered in a college freshmen seminar course may be useful for evaluating BA approaches in settings that may facilitate prevention efforts. Thus, our findings may be most informative and generalisable to other college samples and potentially students in high school. However, it is unclear the degree to which our findings may generalise to clinical samples and settings. Given that BA was originally developed to treat clinical disorders, our findings are limited regarding their generalizability and potential utility for clinical treatment settings. Thus, more research is needed to evaluate whether the format of daily monitoring forms may impact a client's engagement with monitoring during BA treatment for depression and substance use disorder. Furthermore, most evidence for the clinical efficacy of BA is predicated on the use of intensive daily monitoring forms that employ the hourly reporting format; thus research is also needed to determine if the brevity of the form may impact the degree of treatment efficacy. Thus overall, while our work contributes to the literature on BA treatment engagement, more work is needed across a range of samples and contexts, particularly among clinical samples engaged in treatment for depression and/or substance use disorder.

The study had several limitations. First, participants were young adults and mostly white; therefore, generalizability to older populations and racial and ethnic minority populations may be limited. In addition, the study was a pilot trial with a smaller sample (N = 59); therefore, future research should replicate our findings in a larger trial. Furthermore, there were more participants in the focus groups from the intensive monitoring condition compared to the brief condition, and the degree to which brief monitoring participants’ feedback was representative of other brief monitoring participants’ experiences is unknown. However, the intensive monitoring condition was anticipated to create a greater response burden on participants, and representative feedback from intensive monitoring participants was most necessary for comparison. Thus, although the representation of participants was unevenly distributed across focus groups, feedback from students in the intensive condition was likely sufficiently representative to facilitate a reasonable comparison with the brief monitoring condition, which was expected to have a lower response burden overall. Finally, the study examined the completion of activity monitoring overall and did not investigate aspects of the activities that were completed. Thus, future work should investigate in greater detail what type and quality of activities are completed (e.g., whether completed activities are primarily related to values), which may yield insights into factors that may contribute to treatment effects.

Conclusions and Implications

The current pilot study was the first to test whether differences in the format of activity monitoring forms may influence participant engagement with daily monitoring during a BA intervention. Overall, there were no significant differences in the frequency of monitoring or the duration of monitoring engagement between participants in the brief versus intensive monitoring conditions, suggesting that either format may be useful during a BA intervention for college freshmen. No substantive differences in the experiences of participants with the brief versus intensive monitoring forms emerged from the qualitative interviews. However, qualitative findings overall indicated that monitoring may have been particularly useful in the first month, while students adjusted to the college setting and established new living patterns on campus. Thus, results indicated that participants may find utility in monitoring during the first month of a BA intervention using either brief or intensive monitoring forms.