Assessing the Effect of Breakout Sessions on Student Success and Satisfaction

Abstract

Introductory American government is a common component of college and university core curricula and, as such, it often is taught in large sections. This makes active learning more difficult, which may contribute to student dissatisfaction and lower levels of student achievement. In turn, this can affect larger issues of university retention. This article considers whether different models of instruction in large classes affect student success and satisfaction. We compare a lecture-only class and one that combined lectures with smaller student breakout sessions. To our surprise, we found that students in the breakout—lecture class were not more satisfied and did not succeed at higher levels as compared to their peers in the lecture-only class. Above all, attendance is the key predictor of student success.

Introductory American government is a common component of college and university core curricula, which provides both opportunities and challenges for political science departments. On the one hand, the requirement ensures the demand for political science classes and facilitates the recruitment of majors. On the other hand, political science faculty may find themselves teaching uninterested and somewhat skeptical students. Having to perennially win over disgruntled math and fine arts majors can be a pedagogical challenge and may adversely affect instructor evaluations. The current era of constrained state budgets ¹ makes the situation even more challenging. Political science departments facing a mandate to offer university-wide service classes may respond by offering increasingly larger sections of the mandated course. This makes active and collaborative learning more difficult—although not impossible (Huerta 2007).

Offering large sections of introductory American government is a practical necessity. This article considers whether different models of instruction in large classes affect student success and satisfaction. Excellent work already has been accomplished in this area of research (Huerta 2007; Monks and Schmidt 2010; Velasco and Çavdar 2013). Our contribution is to consider the effect of incorporating smaller “breakout sessions” into traditional lectures. Breakout sessions provide opportunities to implement active-learning techniques and create a less intimidating instructional environment. To test whether this contributes to student success and satisfaction, we compared their success and instructor evaluations across two sections of American government, both taught by one of the authors (i.e., Blackstone). One section was lecture only; the other used a combined lecture–breakout format. Our results suggest that the incorporation of breakout sessions into lecture classes may not be a panacea for concerns about student success and satisfaction and, by extension, university retention.

TEACHING LARGE SECTIONS

Class size is negatively related to student perception of learning and instructor evaluation. Instructors often cope with increased class size by modifying their classes in ways that are detrimental to student learning. In particular, large classes seem to undermine student perception of the quality and timing of instructor feedback, an instructor’s respect for students, and the required amount of critical thinking (Monks and Schmidt 2010). Thies (2005, 129) noted that “students learn more and have a more satisfactory college experience when they actively participate in rigorous learning environments, embedded in strong communities, characterized by high levels of interpersonal interaction with their peers and faculty members.” The key, then, is to create this type of environment in large classes.

Thies (2005) argued for the efficacy of learning communities in which classes with the same students are linked thematically to provide greater depth and interaction. Others focus on active-learning techniques. As opposed to traditional lectures, in which students passively receive information from their instructor, active learning emphasizes “student activity and engagement in the learning process” (Prince 2004, 223). Techniques include simulations (Ciliotta-Ruberty and Levy 2000; Hamner, et al. 2015; Krain and Lantis 2006); role playing (Hensley 1993; Larson 2004); student-response technology (Elicker and McConnell 2011; Evans 2012; Velasco and Çavdar 2013); pairing and sharing; and group-based activities. The use of active-learning techniques is not especially widespread in political science gateway classes, such as introductory American government (Archer and Miller 2011). Implemented properly, however, active-learning techniques can bring the benefits of small-scale teaching environments to large-scale environments, and there is growing evidence that use of active-learning techniques enhances student learning and satisfaction.

Huerta (2007) observed that much of the literature on active learning is focused on upper-level classes. This is probably due in part to the organizational difficulties that inhere in managing larger introductory classes, but there are notable exceptions. Frederick (2002) provided excellent guidance on the incorporation of active-learning techniques into large sections, such as associational brainstorming and the use of evocative images. Larson (2004) developed a role-playing activity that easily could be used in a large section because much of the work takes place outside of class. Ishiyama and Watson (2014) demonstrated the utility of a web-based interactive platform for use in large classes designed to improve writing. Huerta (2007) incorporated several active-learning techniques into large sections of state and local government courses. Techniques include small-group discussions, student-generated study guides, and question-based lectures. In comparing his “active” and “non-active” sections, Huerta found that in several instances, students in the active sections achieved significantly higher exam grades. Moreover, students in active sections had more favorable evaluations of their learning opportunities and were more likely to report that they would recommend the instructor to peers.

IMPLEMENTING BREAKOUT SESSIONS

The setting for this study was the University of North Texas (UNT), a public university approximately 40 miles north of Dallas. Two factors are important: (1) students receiving a bachelor’s degree from a public university in Texas are required by legislative mandate to complete six semester credit hours in American government; and (2) the undergraduate enrollment at UNT is about 29,000 students. Although some students enter the university already having completed the political science core requirement at another institution, it can be difficult to provide a sufficient number of classes to meet student demand. Therefore, offering large sections has become standard operating procedure. In the fall of 2014, for example, the political science department offered 19 sections of the mandated courses. One section had an enrollment cap of 700, three had an enrollment cap of 500, and six had an enrollment cap of between 200 and 320 students. This was not an atypical semester.

Given growing concerns for retention and student success—and wanting to ensure that introductory American government does not become a choke point in the curriculum—the department has experimented with different classroom models.

The model at UNT has been to teach introductory American government in a lecture-only format. Given growing concerns for retention and student success—and wanting to ensure that introductory American government does not become a choke point in the curriculum—the department has experimented with different classroom models. In the fall of 2014, we piloted a section taught with breakout sessions. On a weekly basis, the entire 120-student class met for a 110-minute lecture on Mondays. (A 10-minute break was taken midway through each session.) Students then attended one of three 50-minute breakout sessions later in the week. Each breakout session included 40 students and was led by a graduate-student teaching assistant. Students were required to attend the same breakout session each week, which allowed us to leverage the benefits of large- and small-section learning environments (Pollock, Hamann, and Wilson 2011). The breakout–lecture section was compared to a lecture-only section of introductory American government taught by the same instructor in the same semester. The lecture-only class included two 80-minute sessions each week.

The faculty instructor determined what would be covered in breakout sessions, which were designed to complement what had been taught in the lecture. Lesson plans included but were not limited to a discussion of liberalism and American political culture, a discussion of state-level marijuana legalization and its implications for federalism, a one-day legislative-process simulation, and a short lecture on the Texas constitutional-amendment process. To help students get to know one another, in the first breakout sessions, they had to complete a scavenger hunt in which they had to find other students in the class who met certain criteria (e.g., “is not a freshman,” “has served in the military,” and “wants to go to law school”).

We compared student success and satisfaction in the breakout class to a traditional lecture-only class of the same size and taught by the same instructor. The protocols for the experiment were approved by UNT’s Institutional Review Board. The instructor made identical remarks in both classes to explain the purpose of the study, and then students were asked to submit an informed-consent form—irrespective of whether they opted to consent. The form was returned in a sealed envelope with the student’s name written on the outside. All students returning the form received one point on their final grade, regardless of whether they provided consent. Envelopes were not opened until final grades were submitted. In all, 81.7% of students provided consent.

ANALYSIS

Our results are not what we expected. First, we compared student satisfaction in the lecture-only versus the lecture–breakout sessions. Student satisfaction was operationalized using the department’s instructor evaluation instrument. Instructors in the political science department were evaluated by students on 17 dimensions. Evaluations were conducted on a 1-to-7 point scale, with 7 representing excellence. Figure 1 compares aggregate performance on all dimensions. The instructor received favorable evaluations under both formats; however, on almost every dimension, the students gave a more positive evaluation in the lecture-only class. The exception was item 5, “Stimulation of Interest.” ²

Figure 1 Instructor Evaluations

Note: Instructors in the political science department are evaluated by students on the following 17 dimensions, and evaluations are conducted on a 1- to 7-point scale, with 7 representing exceptionally good performance: (1) Organization of the Course; (2) Knowledge of the Subject; (3) Preparation for Class; (4) Enthusiasm for the Subject; (5) Stimulation of Interest; (6) Speaking Ability; (7) Ability to Explain; (8) Tolerance of Disagreement; (9) Attitude toward Students; (10) Availability for Conferences; (11) Quality of Testing; (12) Fairness in Grading; (13) Quality of Instructional Materials; (14) Promptness in Returning Assignments; (15) Responsible Classroom Conduct; (16) Intellectually Challenging; and (17) Overall Evaluation of Instructor.

Second, we considered measures of student success. In UNT parlance, introductory American government has a very high DFW rate, which means that an unusually high proportion of students receives a grade of D or F or withdraws from the class entirely. This delays students’ progress toward their degree and it may impact the university’s overall retention rate. We expected to find a lower DFW rate in the lecture–breakout class because students would have had a greater opportunity to get to know and learn with their peers. To the contrary, we found that the DFW rate was somewhat higher in the lecture–breakout section: 21.8% compared to 20.7%. ³

We ran several t-tests comparing overall course grades, exam grades, and homework grades between the two classes. The results, which are provided in table 1, were consistent with our blunter consideration of the DFW rates. With respect to homework grades, the classes were indistinguishable. With respect to exam grades and overall course grades, students in the lecture-only section performed better. Mean exam grades were almost five points higher in the lecture-only section, whereas mean overall course grades were about four points higher. These differences achieved statistical significance at the 0.05 level.

Table 1 Comparing Mean Student Outcomes

Note: *denotes statistical significance at the 0.10 level, ** at the 0.05 level (two-tailed tests).

Table 2 displays four models that analyze student success in more detail. For all models, the unit of analysis is the student. In models 1 and 2, the dependent variable is the “Course Grade” on a 0-to-100 scale; in models 3 and 4, the dependent variable is the “Mean Exam Score.” These dependent variables were regressed on several independent variables that are likely to contribute to student success. “Breakout Class” is an indicator variable that distinguishes students in the lecture-only section from the breakout–lecture session (breakout = 1). “Attendance” measures the proportion of time that students were in class when attendance was taken. “Freshman” is an indicator variable that distinguishes between freshmen and sophomores, juniors, and seniors (freshman = 1). The “Homework” variable is included in model 4 to reflect the proportion of homework questions completed.

Table 2 Regression Analysis of Student Success

Notes: *denotes statistical significance at the 0.10 level, **at the 0.05 level (two-tailed tests). Standard errors clustered by teaching assistant in parentheses.

Models 2 and 4 also include several demographic variables. Demographic data were collected using a voluntary survey on Blackboard. “First Generation” is an indicator variable that measures whether students identify as a first-generation college student (first generation = 1). “Employment” reflects the number of hours per week that they work. “Language” is an indicator variable that distinguishes between students whose spoken language at home is not English from their peers (language at home other than English = 1). “Female” is an indicator variable based on sex (female = 1).

Again, students in the lecture–breakout section did not outperform their peers in the lecture-only section—indeed, they did worse. Controlling for other factors, lecture–breakout students had course grades and exam averages that were at least two points lower than their peers in the lecture-only section. This negative relationship achieves significance in models 1, 2, and 3 and it approaches significance in model 4. As we might expect, attendance was positively related to student success. The variable ranges from 0 to 1; therefore, the coefficient reflects the expected difference in grade between students who never and those who always attended class. As expected, the difference was pronounced. Those who never attended class earned course grades between 11 and 18 points (model 2 and model 1 estimates, respectively) lower than those with perfect attendance. Of course, most students’ attendance records fell between the two extremes. Each week’s worth of attendance translates to between 0.89 and 1.4 additional points (model 2 and model 1 estimates, respectively) on a student’s course grade.

DISCUSSION AND CONCLUSION

As noted in the introduction, the incorporation of breakout sessions into lecture classes may not be a panacea for concerns about student success and satisfaction and, by extension, university retention. We expected that the breakout–lecture format would facilitate engagement with students’ peers and the instructor, thereby boosting their success and satisfaction. However, we found that the opposite is true. ⁴ Above all, we found that attendance is positively related to student success.

What explains these findings? There may be an issue with our design. First, we acknowledge that the experiment would have been more controlled if the faculty instructor had led the breakout sessions rather than graduate-student teaching assistants. The teaching assistants did an excellent job, but the faculty instructor teaching would have provided a better basis of comparison between the two classes. Second, the breakout sessions may have been too large to enhance student success and satisfaction. We suspect that 20-student breakout sessions may yield different results. At the same time, having a faculty instructor lecture and lead breakout sessions almost certainly would be impractical in departments that want to develop and expand the use of breakout sessions. Furthermore, 20-student breakout sessions would produce tremendous logistical difficulties at UNT as well as at other institutions with comparable enrollments.

We expected that the breakout–lecture format would facilitate engagement with students’ peers and the instructor, thereby boosting their success and satisfaction. However, we found that the opposite is true. ⁴ Above all, we found that attendance is positively related to student success.

Third, we note that students were not randomly assigned to treatment and control groups. As a result, there may have been systematic differences between the two classes that undermined the experiment. Descriptive statistics demonstrated that the lecture-only class had a lower proportion of freshmen and first-generation college students. Those variables did not significantly affect student success and satisfaction in the empirical analysis, but the lecture-only students may have been more experienced and therefore in a better position to succeed. ⁵ Moreover, the lecture-only section met twice per week for 80 minutes, whereas the breakout–lecture section met for almost 110 minutes of lecture (to facilitate the scheduling of breakout sessions). Anecdotally, students reported fatigue and difficulty staying focused in the second hour of the lecture, which may have driven down attendance. Were the lecture–breakout class scheduled differently—for example, two 50-minute lectures and a 50-minute breakout session—the results may have been closer to our original expectations.

Despite these limitations and our counterintuitive findings, we argue that our results are valid and reliable. The breakout–lecture format is associated with stimulation of interest but not overall success and satisfaction. Students may not learn more, but this finding suggests that the lecture–breakout format stimulates curiosity and engagement—a key objective of active-learning strategies. We speculate that our results may be attributable to a combination of factors associated with the ways in which students approach breakout sections. It is possible that they simply take non-instructor–led activities less seriously than lectures delivered by the instructor of record. Alternatively or additionally, students may have difficulty determining what they are meant to glean from more-interactive sessions. Finally, we are convinced that attendance is key. Students in the lecture–breakout section were significantly more likely to skip the weekly lecture than their breakout sessions, and lecture attendance was significantly higher in the lecture-only format class. As a result, students in the lecture–breakout sections had, on average, even fewer contact hours with the faculty instructor than the design necessitated.

Our results suggest that incorporating active-learning techniques into large sections (Huerta 2007) may have more utility than adopting new class formats. Given the concerns for external validity in experimental designs, however, we are not ready to dismiss the usefulness of the lecture–breakout format. We look forward to using it again in the future and more fully exploring student attitudes toward different modes of instruction.