“We're Off to Replace the Wizard”: Lessons from a Collaborative Group Project Assignment

Abstract

This article examines the effectiveness of a collaborative group learning project for teaching a core competency in comparative politics: constitutional structures. We use a quasi-experimental design and propensity score matching to assess the value of a constitutional writing group project and presentation. The results provide strong evidence that these learning tools are highly valuable for teaching abstract concepts. Students who participated in the project scored significantly higher on a short series of questions in final exams given several weeks after the completion of the group project. Somewhat paradoxically, the project increased competency but did not affect student self-reported interest in the subject matter. The challenges and improvements that can be made for the use these types of learning tools concludes the article.

Questions about how to improve student understanding of core concepts drive a growing interest in pedagogical innovation in undergraduate political science education. This article explores one such approach: collaborative group project assignments. Using a quasi-experimental design, we empirically test whether this approach improved student learning and, if so, by how much. In discussions about various kinds of “active,” “cooperative,” or “team-based” learning, arguments focus on why these approaches better “engage” students or otherwise improve the educational experience (Johnson, Johnson, and Smith 1991; Michaelson, Knight, and Fink 2002; Millis and Cottell 1998). These approaches embrace a range of pedagogical styles, including “active reading” (Daley 1995), using film or other audiovisual components to “enhance” learning (Ulbig 2009), and engaging students through blogging and social media tools (Lawrence and Dion 2010). A much larger literature focuses on simulations.¹ This article looks at one example of cooperative and collaborative learning: a small and underdiscussed subset of the “active” learning pedagogy. To date, we have found only one reference to the use of cooperative learning in political science (Occhipinti 2003), which offered a descriptive guide to its pedagogical application. Although we have used this (and the broader “cooperative” or “collaborative” literature) as a guide, this article is particularly interested in empirically testing a core claim made by proponents of cooperative and collaborative learning: cooperative or collaborative learning activities improve student learning.

Our test emerged from a particular interest in determining whether a specific collaborative group project assignment (which one of the authors had previously used) was an effective tool for learning differences in democratic political institutions. The assignment—in which students, working in small groups over several weeks, draft a constitutional framework for a fictional country—had been used in two previous upper-level courses on democratization. Those experiences suggested collaborative assignments helped students develop a working understanding of the range of institutional engineering choices available to emerging democracies. We were interested in testing whether the assignment could be adapted to an introductory-level course in comparative politics at a large public university. In our experience teaching introductory courses, students have difficulty understanding unfamiliar democratic institutional systems, such as unitary systems, proportional representation, or other forms of nonplurality voting, as well as parliamentary or semipresidential systems. We hoped that by using a collaborative learning assignment students would master these difficult concepts. We planned to test whether the assignment met that goal.

COOPERATIVE TEAM LEARNING AND COLLABORATIVE LEARNING

Our collaborative group project fits within the broader categories of “active” and “experiential” learning (Johnson, Johnson, and Smith 1991; Michaelson, Knight, and Fink 2002; Millis and Cottell 1998). The term “active learning” describes any teaching approach that transcends the traditional lecture environment, which is frequently described as a “passive” mode of learning. Active learning pedagogy describes a variety of techniques, such as staging classroom debates, fostering small group activities, or making students engage in illustrative “games.” The term “experiential learning” describes approaches that take active learning further by providing students with experiences that either mirror “real world” conditions (simulations) or offer direct experience in a real world setting (internships, field-site activities) or shift student attitudes or perceptions by exposing them to new environments (often a key element of “service” learning). In terms of pedagogy, both approaches engage students in learning through practical, “hands-on” application of material and concepts. Within these two broad categories are two specific pedagogical approaches that could describe our project: cooperative team learning and collaborative learning. Although the project we describe here more closely fits the cooperative team learning approach than the collaborative learning approach, we call our project a “collaborative group project” because of significant distinctions from both approaches.

Cooperative team learning is “highly structured” and “entails positive interdependence and student accountability” (Occhipinti 2003, 69). Unlike traditional kinds of small-group, in-class activities, a key distinguishing feature of cooperative team learning is that groups work together over a longer period. Additionally, students are assessed (i.e., graded) both collectively (which fosters interdependence) and individually (which fosters accountability). Both elements are critical: collective assessment requires students to cooperate and help each other master learning goals and successfully execute projects; individual assessment reduces the incentive for students to become “free riders.” Another key feature of cooperative team learning is that teachers closely monitor groups and provide requisite “social training” or some framework of rules and group roles. In many ways, cooperative team learning is an extended form of group learning, but with static group membership that extends beyond a single class period (whether only a few weeks or a full semester).

Collaborative learning, in contrast, specifically demands limited teacher guidance to allow groups to interpret, question, challenge, or even critically “break down” larger frameworks of understanding. Derived from feminist pedagogy, collaborative learning allows students to interpret, critique, deconstruct, or create their own frameworks of understanding. This approach is well suited for engaged, critical thinking—particularly when the goal is to tackle more normative, subjective, or theoretical issues, such as justice, freedom, and democracy.

Cooperative forms of learning are well suited to help students master relevant material, functioning as “study groups.” The assumptions underlying cooperative learning are that students share responsibilities and help each other master common material using consciously constructed incentive structures (collective assessment) to ensure that stronger students help weaker students. Collaborative learning is better suited to develop students' critical thinking skills because students work on projects that do not have a priori “solutions.” In addition, by requiring limited teacher oversight, collaborative learning fosters student independence and provides space for student exploration.

We designed our project to blend some of the assumptions and goals of cooperative and collaborative learning approaches. We wanted students to work in groups to collaborate on a project with no objective “solution” (there was no “correct” constitution for our fictional country). Thus, students brought their own normative values or assumptions about human nature to the group, which then collectively developed an understanding of the assignment. However, we did not want our students to stray too far from the basic concepts we wanted them to understand (executive type, electoral system, federal-unitary model). Thus, we expected our students to work together to “learn” how to distinguish between institutional models.

THE ASSIGNMENT: A CONSTITUTIONAL DESIGN FOR OZ

In our collaborative group project students worked in small groups to draft a constitutional framework for the Land of Oz.² Each group consisted of five to seven students and had three to four weeks to work collaboratively to produce two final products: a short paper (six to seven pages) and an oral presentation (7 to 10 minutes) to the class. Students organized their group's efforts independently with little direct involvement from their instructor. Students knew that each group's final paper and presentation would receive a grade, which would become the “base” grade for each group member.³

Although both instructors were experienced faculty members we chose two treatment courses because of their structural differences.⁴ We wanted to see if the group assignment would have similar effects on two otherwise structurally dissimilar courses. Class A was a midsized lecture class with 54 students; Class B was a composite class of on-campus students (24) and off-campus students (7) who met simultaneously via video link. Both classes met twice a week and were primarily lecture-driven with some discussion. Additional similarities and differences are listed in table 1.

Table 1 Comparison of Comparative Politics Classes

The project began midway through the semester: students were placed randomly into groups and given the assignment's guidelines. Then, students received a dossier that included a map of Oz and a brief description of the fictional country's political geography, demographic divisions, and recent history.⁵ We made it clear that students were expected to use the information in the dossier and refer to it throughout the project. Their task was not to design an “ideal” constitutional framework, but rather one that was tailored to the needs and realities of Oz.

Students were required to specifically consider three primary questions, each of which had to be addressed in the final written and oral reports:

1. (1) What kind of executive system should Oz adopt (presidential, parliamentary, or semipresidential)?

2. (2) What kind of electoral system should Oz adopt (first-past-the-post, proportional representation, or other)?

3. (3) Should Oz adopt a federal or unitary system?

In addition to these questions, students could address other issues they thought important. Students addressed a wide variety of issues. Although these were included in our overall assessment of their group product, we were primarily interested in the three primary questions, on which we focus in this article.

Although we adopted different textbooks for our respective classes, both texts covered these fundamental topics. Class A used Patrick O'Neil's Essentials of Comparative Politics, which covers all three topics in Chapter 5 (“Democratic Regimes”). Class B used Carol Ann Drogus and Stephen Orvis's Introducing Comparative Politics, which covers the topics in Chapters 6 (“Political Institutions: Governing”) and 7 (“Political Institutions: Participation and Representation”). Despite their differences, both texts provide similar discussions of electoral systems, executive type, and federalism to give students a baseline for their research.

Each of us also pursued a slightly different pedagogical approach throughout the semester. Class A combined the textbook with the ancillary Cases in Comparative Politics (O'Neil et al. 2009), covering 13 different countries during the semester, and in-class economic games for teaching basic political economy concepts. Class B combined the textbook with a series of 10 thematic additional “texts” (including audio podcasts, video, and online interactive content) that were paired with short analytical writing assignments (students chose to write on any three of the “optional” assignments, plus one required assignment, for a total of four).

ASSESSING THE PROJECT'S EFFECTIVENESS: A QUASI-EXPERIMENTAL DESIGN

The primary goal of the collaborative assignment was for students to learn about three key dimensions of democratic institutional design: executive-legislative relations, electoral system, and federal-unitary structures. Optimally, we hoped students would complete our classes with an understanding of the costs and benefits of each system. Although students received a grade based on their collaborative effort, we hoped that—through participating in a collaborative project assignment—individual students would meet our minimal expectations of correctly identifying different institutional systems. For measurement validity across three classes, we focused on an assessment of minimal learning outcomes. However, anecdotal evidence suggests that a range of students were able to articulate important differences between different institutional systems, as well as the implications of such differences.

The final exam was our primary instrument for assessing whether students met our minimum expectations. Although each exam reflected material covered in our respective classes, we included four identical multiple-choice questions that specifically asked students to identify different institutional systems along our three dimensions (see the appendix).

To determine whether our collaborative assignment enhanced student learning, students in another class (C; taught by an experienced graduate student instructor under the guidance of one of the authors) answered the same four multiple-choice exam questions as part of their final exam. Although these students did not participate in a collaborative project, they completed the same confidential pretreatment survey as the two “treatment” classes (A and B).⁶

We expected students to perform significantly better in the “treatment” classes. Using three classes allowed us to control for potential instructor, textbook, and pedagogical effects (see table 1) unrelated to the collaborative assignment.⁷ Although it is possible that instructor experience played a factor in our results, two reasons suggest that this was not the case: First, exam and end-of-semester grades for all three classes were generally consistent, suggesting that students completed all three courses with similar mastery of material in other areas of the course. Second, we found no evidence that student evaluation of “teacher quality” had any effect: Both midsized enrollment classes (A and C) had similar teaching evaluations, suggesting differences in student performance on the test indicators were not due to teacher quality effects. Similarly, Class B had significantly different teaching evaluations from Class A, yet Class B showed no similar difference in student performance on the four indicators.⁸

We also controlled for independent variables that could affect our dependent variable (performance on the “institutional” final exam questions) across individual students. Thus, we also distributed a pretreatment questionnaire to all three classes. Although the survey was not anonymous (we needed to match individual survey responses to assignment and test performance), it was confidential. Furthermore, as we told the students, we did not review the survey until after the end of the semester and final grades had been submitted. Our pretreatment questionnaire included demographic questions (age, race, gender, socioeconomic class), nonstandard demographic questions unique to educational environments (disciplinary major), and questions designed to tap into various attitudinal dimensions (ideology, trust, and social communication). See table 2.

Table 2 Demographic Indicators across Comparative Politics Classes

Finally, we were also interested to learn what students thought about working in our collaborative project along different dimensions: student behavior (how often the group met, how many hours were spent on the project, how was work distributed among group members), attitudes towards the project (what aspects of the project were most enjoyable or difficult, what would students change), and whether participating in the collaborative assignment changed interest in the course. We assessed this last dimension by asking students about their level of interest in “comparative politics and the subjects we cover in class.” We also included this question in the posttreatment survey, allowing us to see if attitudes shifted after participating in the collaborative project. Because students in class C did not participate in the collaborative project, they did not take this second (posttreatment) survey. Although this approach limits our ability to explore differences in the interest-level dimension between the treatment and nontreatment cases, we were primarily interested in changes across a semester from a baseline interest in the subject. We hoped that a “fun” collaborative assignment near the end of the semester would generate interest in comparative politics.

ASSESSING THE PROJECT'S EFFECTIVENESS: THE EVIDENCE

Evidence suggests that the collaborative assignment helped students learn about different political institutional designs. On average, students in the treatment classes scored better on the four final exam multiple-choice questions about institutions than students in the nontreatment class—and that difference was statistically significant (table 4).⁹ Additionally, we found no significant difference in test scores between the two treatment classes, suggesting that textbook and pedagogical approaches had no significant impact on student performance.

As table 3 illustrates, students who participated in the group project scored dramatically higher on all four questions from the final exam. The smallest gap between the treatment and nontreatment groups is on the executive type question, yet this gap is still 35 percentage points. The majority of those in the treatment group answered all four questions correctly yet only 12% percent of the nontreatment group did so. As table 4 shows, after taking into account differences between the two groups, the average number correct in the treatment group (Cronbach's α = .76 for both groups) was 3.6, however, in the nontreatment group the number was only 1.1. Participating in a collaborative group project had a significant impact on students' competency of fundamental topics often at the center of comparative politics.

Table 3 Student Performance on Institutions Exam Questions across Classes

Table 4 Treatment Effect of Project Using Propensity Score Matching (N = 99)*

* Average number of correct questions does not match figures in table 3 because of the propensity score matching procedure, which drops outlier cases. Subjects were matched on final class grade, gender, parental education level, hours studied per week, race, and interest in the subject matter.

The project appears to have increased student's understanding of differences in formal constitutional structures, however, this improvement was not uniform. Although the treatment improved students' test results for each of the questions, the effect size was smallest for the first question dealing with executive type. This indicates that American students are likely to have difficulty in understanding the nuances between presidential and parliamentary systems, in particular the differing sources of executive authority/constituency.

An interesting advantage of a posttreatment survey is to review what changes to students' interest in the subject matter occurred, if any, due to the project. Our results indicate that although the treatment had a clear and dramatic effect on students' understanding of constitutional systems, it had no effect on students' interest in comparative politics. Nearly every student rated their interest in the subject matter at the same level in both the pre- and posttreatment surveys.¹⁰ There was a clear disconnect between the treatment's ability to increase understanding and interests in the subject matter. Again, we saw no effect of student evaluations of “teacher quality” on student interest. Despite a significant difference in student evaluation scores between the two treatment classes, students left the courses with similar interest (or disinterest) in the subject matter.

The posttreatment survey also allows us to examine which students enjoyed the project and what common difficulties students faced in completing the project. In general, students who reported studying more hours per week and had higher grades in the classes were the students who were the least satisfied with the group's product (Pearson's r = −0.23, p < .05). Similarly, students who said someone in the group did more of the work least enjoyed the project (Pearson's r = 0.45, p < .01). These correlations hold if we look at the group average level (Pearson's r = −0.7, p < 0.01 and Pearson's r = −0.52, p < 0.05, respectively). This suggests that the collective nature of the group project—and the collective action problem groups inherently generate—seemed to cause better students to view the quality of the product produced and how the group functioned more negatively. Anecdotal evidence shows that “good” students felt over-burdened and stressed during the project, particularly if one or more group members did not actively participate. To some degree, the postpresentation quiz that allowed students an opportunity for intragroup peer critique helped alleviate some students' concerns regarding issues of equal work—as did our grade adjustment procedure.

Finally, we anticipated that students who expressed higher levels of comfort communicating in social situations (as measured by scale from a battery of questions, Cronbach's α = 0.82) would benefit the most from the treatment. Although higher levels of comfort with social communication were correlated with satisfaction with group product (Pearson's r = 0.23, p < 0.05 at the individual-level; Pearson's r = 0.56, p < 0.01, at the group-mean level), comfort levels were not correlated with the measured learning outcomes.

CONCLUSION

The evidence supports our claim that our collaborative group project assignment was successful to teach core concepts of institutional design to students in introductory-level course in comparative politics. However, we were surprised to find that participating in such a project had no significant effect on students' self-reported interest in political science. Proponents of “active” modes of learning often claim that such approaches better “engage” students. In particular, proponents of “cooperative” and “collaborative” learning strategies often have (implicit or explicit) goals of developing core “civic” values. Our findings suggest that these claims may not hold. We did find that students better learned course material using collaborative group project; but we found no evidence that the experience carried beyond the immediate context or affected student attitudes toward politics or learning.

Based on our reflections over the strengths and weaknesses of the project, we offer the following advice for others attempting similar projects:

1. (1) Assign groups earlier and use them regularly to encourage students to prepare for other class assignments. This is closer to Occhipinti's (2003) “collaborative team learning” model; this approach might also foster greater bonds of trust between group members in a low-stakes setting prior to working on a much larger graded project.

2. (2) Set aside class time for groups to meet and work together. One common complaint raised by students was the difficulty of finding time to meet together in their groups. This in-class group time would also allow the instructor to meet with each group, even if briefly, and address any questions, issues, or potential conflicts.

3. (3) Assign specific—but focused—reading assignments on the core concepts and give periodic quizzes throughout the projects on a specific concept (e.g., executive type and executive-legislative relations). Periodic individual- and group-level assessments (holding groups accountable for the performance of their members) can offer metrics for progress, help clarify questions before the final assignment, and provide regular goalposts for groups (and help avoid group procrastination).

4. (4) Finally, ask each student to write a brief (one-page) reflection to submit with the final group project. This encourages student accountability, gives each student “ownership” of the project, and provides a venue for any dissatisfied group members.

Cooperative and collaborative learning strategies are effective in helping students learn course materials. From our collaborative group project assignment the evidence suggests a significant boost in student performance after participating in such projects. This result is particularly remarkable considering that our final exams were given nearly a month after the group project was completed. We encourage other colleagues to develop assignments that incorporate these pedagogical approaches.