Rubrics in the Political Science Classroom: Packing a Serious Analytical Punch

Abstract

Although professors may be aware that rubrics shorten grading time and improve grading consistency, many are not aware that rubrics offer a powerful analytical punch. Given the demands for active learning in today's college classroom, rubrics allow instructors to focus on analytical quality while engaging students in a variety of assignments. Rubrics are useful not only in more traditional applications—for example, papers and oral presentations—but also for more creative purposes. Using rubrics in both self and peer assessment engages students more in assignments, allowing them to reflect on their own performance and their peers. Going one step further, instructors can engage students in the construction of rubrics that will be used to grade their own work. Finally, rubrics force instructors to be clear about their own purposes for an assignment, and over time, instructors can become more attuned to the analytical possibilities in even traditional “busywork” assignments.

The political science classroom today can be a busy place: simulations, debates, wikis, and service learning on top of regular reading, lectures, exams, and assignments. With the current emphasis on “active learning” in the classroom, the pressure is on the instructor to ensure both diversity and quantity in assessment. The problem is, if one is not careful, many active learning assignments can turn into busy work—or at the very least, fail to push students intellectually. In the midst of all this, how can one ensure quality as well? The use of rubrics can help professors design and grade a variety of assignments while keeping the focus on analytical thinking. As Young argues,

Effective rubrics based on quality criteria can prevent a downward spiral into busywork and crafty ideas. We can design solid assignments that address real imaginative and creative thinking, allowing the students to explore concepts beyond the standard level, moving them to develop the abilities and skills we want them to take into the real world

(2009, 76).

In the political science classroom, rubrics can help college students rise above mere description and speculation and learn how to analyze material and create their own insights in quality research products. Helping students gain these abilities is critical given that our discipline is marketed to students based on the analytical and communication skills they will acquire. Analytical writing skills are especially important; professors have expressed concern about declines in student writing skills, and business groups have lamented the lack of writing ability in employees (College Board 2004; Dillon 2004). Rubrics allow instructors not only to clearly spell out the technical requirements for assignments, but also to signal to students the importance of analysis and research. This article examines the main issues related to using rubrics to enhance analytical learning: their use, benefits, and costs. It also gives examples how rubrics can be used in the college classroom, not only to allow easier assessment of the myriad of assignments, but also to teach higher-level analytical skills on which academia places a premium. Finally, creative uses of rubrics can enrich student learning and analytical capabilities.

ISSUES IN RUBRICS USE

What Are Rubrics Used For?

Although rubrics can vary widely according to purpose, length, and detail, all rubrics share the core feature of serving as “an assessment tool that lists the criteria for a piece of work and articulates gradations of quality for each criterion” (Andrade 2005, 27). Rubrics were originally designed to assess written composition (Popham 1997, 72), but now these are used in a wide range of fields, from nursing to literature, chemistry to communications. In addition, rubrics are now used to score not only writing products, but also oral presentations, video production, graphic design, debates, wiki contributions, and more. Found all the way through primary school to postgraduate education, rubrics are normally used by the instructor or grader to assess work; however, rubrics can also be used by the student for self-assessment, or by peers for peer evaluations.

Within political science, rubrics can be applied to a wide variety of tasks. The first, of course, is to assess written products; rubrics can be created to judge analytical summaries, policy memos, futures scenarios, research papers, and more (Pennock 2011; Roever and Manna 2005). Presentations are also amenable to scoring with rubrics (Sum and Light 2010). For example, an assignment for an analytical briefing could use a rubric that would emphasize the importance of original insights through categories devoted to analysis and argumentation (as opposed to merely summarizing facts). A notoriously hard task to measure—participation—can be made easier with rubrics, whether for class participation or participation in a simulation, such as Model United Nations (Taylor 2011). Debate performance is another area ripe for rubric use, as are group projects (because one criterion can be individual contribution).

The Analytical Benefits of Rubrics

Rubrics offer numerous administrative benefits of interest to political science instructors: they shorten grading time, improve grading consistency, and can increase student satisfaction (Roever and Manna 2005; Stevens and Levi 2005). What many instructors may not realize is that rubrics also provide substantive benefits in helping to increase the analytical focus in assignments.

First, rubrics force professors to clarify goals and expectations. What makes a good paper? To create a rubric to grade papers, that question first must be asked and answered in detail. As Andrade notes, “Rubrics orient us toward our goals as teachers” (2005, 27). More specifically, rubrics force us to “systematize on paper what teachers often attempt to do in their heads: establish evaluation criteria for learning and apply the criteria to individual work” (Montgomery 2000, 325). By forcing instructors to articulate standards, rubrics discourage grading from a sense of “I'll know it when I see it.”

As a result, a carefully constructed rubric is more than just an assessment tool: it is also a useful teaching tool. Because rubrics force clarification of expectations, they make creating instructional guidelines easier. Explaining these expectations to students is much simpler with a rubric because objective criteria already have been laid out. Thus, the quality of instruction may improve because the rubric helps provide clear-cut directions on what defines an excellent product. Research shows that students appreciate the clarification of requirements that rubrics provide (Andrade and Du 2005, 3).

Rubrics can be especially useful in college-level classes because instructors may expect students to know basic requirements of an assignment, when in reality students may not have learned those requirements in high school. For example, a research paper requires a hypothesis, relevant evidence, clear analysis, and sound judgment. But as Stevens and Levi note, “For most professors, these demands are so basic that they are often left implicit in the assignment and so may be overlooked by the students until the assignment is complete” (2005, 22). By making these expectations explicit, professors contribute “to students' development of a more scholarly form of critical thinking—that is, the ability to think, reason, and make judgments based on an independent, accurate accumulation of data and an open-minded approach to each new topic” (Stevens and Levi 2005, 21).

Indeed, one of the most powerful analytical benefits of rubrics is that they can help produce “mindful involvement” on the part of students—without which, student performance can be likened to “a ship without a rudder—in motion, but out of control” (Saddler and Andrade 2004, 49). With clearly articulated expectations, students can informally assess their work as they complete it, which teaches them to take responsibility for judging the quality of their work instead of abdicating that responsibility to the instructor (Saddler and Andrade 2004, 51). For example, when writing a paper, students can use the rubric criteria to self-check as they write.

As noted, one of the most attractive advantages of using rubrics is that they shorten grading time, up to as much as 50% (Stevens and Levi 2005, 17). Indeed, much of the work of grading takes place when the rubric is created—that is when the instructor makes the intellectual judgments about what is expected in an assignment, and the quality levels students should achieve. There are at least two analytical side benefits to simplified grading. First, rubrics allow instructors to assign more complex or challenging assignments if desired (Andrade 2005, 29). Second, shorter grading times mean students receive feedback on their work more quickly, and timely feedback is linked to significant learning benefits. Research has shown that the sooner feedback is received, the more meaningful it is to students and the more likely it is to have an impact on student performance (Black and Wiliam 1998; Rucker and Thompson 2003; Stevens and Levi 2005).

The Analytical Costs of Rubrics

Rubrics offer significant analytical benefits. So why don't more professors use them? In part, this may be due to lack of knowledge about the utility of rubrics or concerns that rubrics should not be used in higher-level or complicated assignments. It may also be, in part, because rubrics carry a number of analytical costs.

First, developing rubrics take time because of the analytical effort it takes to create a high-quality tool. Creating a quality rubric forces the instructor to examine what they believe to be the main evaluation criteria for the work, and then assign levels of quality for each criterion. This is not an administrative task, but rather an intellectual effort that requires time and energy. With the demands of teaching, scholarship, and service, faculty may not see rubric creation as the best use of their time, or even if they do believe rubrics may be useful, they may not have the time to invest in rubric creation.

Another potential cost is inaccurate or poor student learning that results from improperly constructed rubrics. Omelicheva notes, “A scoring instrument will have little instructional value if it is replete with inconsistencies and vague descriptions of performance criteria across its scale levels. The performance criteria are the rubrics' most critical components that require thorough consideration” (2009, 179). Tierney and Simon (2004) argue that “Students do learn from rubrics with inconsistent performance criteria, but what they learn may not be the intended learning goal” (2004). Another way in which rubrics can hurt student learning is by focusing them on the wrong goals. For example, a rubric for a research paper may have more categories dealing with practicalities (such as citation format and adherence to style guide) than intellectual content (such as analysis and evidence). In this case, students may focus more heavily on the shell of the assignment instead of the analytical core (Truemper 2004, 564).

Finally, a possible significant drawback to rubrics is inappropriate student perceptions and use of them. Students may see a rubric as a simple map or recipe to a good grade with the expectation that checking off rubric boxes will automatically result in an A. A well-designed rubric can help avoid this perception because evaluation criteria such as “analytical depth” and “robust, relevant evidence” are not simple boxes to quickly be checked. Nevertheless, research has shown that students do tend to focus on the “A” and “B” descriptors in rubrics (Andrade and Du 2005, 6). Students may also fail to understand that rubrics are designed to describe excellent performance and instead see rubrics as a guide to idiosyncratic instructor demands. Andrade and Du note that “students who talked about a rubric as a map for giving a particular teacher what he or she wanted appeared to have little sense of a connection between their teachers' expectations and a broader definition of quality. We were troubled by these students' uncritical acceptance of criteria and standards” (7). Because one of the most valuable aspects of rubrics is to help students develop internal quality standards, such student perception of rubrics is disheartening.

RUBRICS IN THE POLITICAL SCIENCE CLASSROOM: TWO APPLICATIONS

Analytical Papers

My experiment with rubrics began with a desire to increase both the quality and quantity of analysis in student papers. While I was providing students with the opportunity to write analytical papers, I felt I wasn't doing the best job teaching them how to do so. To correct this, I created an analytical writing rubric for my political science classes. To design the initial rubric, I evaluated my expectations for an analytical paper; specifically, I asked, “What components of a paper created good analysis?” Then I outlined these expectations in the rubric, to clarify for my students and myself what constituted good analysis. See table 1.

Table 1 Analytical Writing Rubric

I found that students improved their analytical skills because of the rubric. Several students commented that the rubric helped them better focus their papers; as one student noted, “I went through the rubric, wrote the paper, looked at the rubric again, and then made a lot of changes.” In addition, the rubric made it easier to explain to students what they did wrong on their papers. If they had an interesting analytical point but failed to develop it with supporting facts and logic, I could easily point out an entire rubric category is devoted to “support for arguments.”

In terms of predicted costs associated with rubrics, the only one that I found was the initial effort that went into creating the rubric. Students did not react to the rubric as an easy-to-check “menu” for a good grade, but rather saw that A-quality work required intellectual energy and attention to writing. In addition, numerous students mentioned that the improvements they experienced in their analytical writing in my class have transferred to higher writing grades in other classes. Thus, students were less likely to see the rubric used in my class as simply a guide to my own idiosyncratic preferences. Finally, although creating the initial rubric did take several hours, it saved hundreds of hours in grading because the rubric streamlines grading. While I still include narrative comments for students, those focus on the substance of the paper rather than issues already covered by the rubric. My experience has been so positive that I have created a number of rubrics for other analytical products, including briefing presentations, film analyses, and policy reports.

Analytical Briefing Presentations

Oral presentations are another staple in the political science classroom, and like papers, can benefit from the use of rubrics. A major problem with college presentations is that some students assume they are similar to those from high school: oral reports that constitute little more than a recitation of facts. A real challenge is getting students to understand that their presentations must be analytical; they cannot simply summarize facts or others' opinions. Instead, they must synthesize material, give critical comments on the readings, and present their own insights on the issues. To combat this, I created an analytical presentation rubric to signal to the students that their presentations should focus on research and analysis rather than description. I did so by modifying the analytical writing rubric, keeping many of the same elements of argumentation, originality, and evidence supported by research—but adding a few categories relating to handout and PowerPoint quality.

Using a rubric for briefings was helpful in several ways. First, it alleviated some of the nervousness students feel with oral presentations. In addition, with clearly defined characteristics and levels of performance, rubrics made grading oral presentations less subjective—both from the instructor and student perspective. My experience is not unique; one project designed to test the use of case studies in teaching students oral communication skills found that rubric use led to considerable inter-rater reliability (Noblitt, Vance, and Smith 2010, 29).

Adaptability

After a professor creates a basic analytical rubric, the ability to adapt it to other assignments is almost limitless, bringing additional benefits. First, over time, a professor will learn to pinpoint immediately the key analytical qualities that are expected from a particular assignment. In other words, professors become more attuned to the specific analytical qualities desired in assignments—and thus can alert them to this and focus on it more thoroughly in their teaching. Second, adapting the rubric to different assignments allows professors to find ways to make nonanalytical assignments more analytical. Busy work can be transformed to analytical work; in my own classroom, I now use rubrics for simulation reflection papers and debate performance to focus students on the analytical facets of those assignments.

GOING FURTHER: CREATIVE USE OF ANALYTICAL RUBRICS

Self Review

Providing rubrics in advance not only allows students to be clear about what is expected of them. Instructors may also ask students to review their own work according to rubric standards, either while students are drafting their paper or as a final evaluation of their work. If turning in drafts is part of the writing process, students can turn in a marked rubric with a draft. Even if formal drafts are not part of the assessment process, instructors can still encourage students to check the rubric as they write their papers. As Saddler and Andrade argue, “With practice, [students] will not only internalize the standards of quality as defined by the rubric, but also develop the habit of self-assessment—a hallmark of self-regulated writers” (2004, 51). In addition, students can be asked to turn in a filled-in rubric, assessing their work, when they turn in the actual assignment.

The benefit of this practice is that it forces students to critically examine their own work against the standards; thus, grades become something that they earn rather than something that instructors “give.” Goodrich advises,

Once you've created a rubric, give copies to students and ask them to assess their own progress on a task or project. Their assessments should not count toward a grade. The point is for the rubric to help students learn more and produce better final products, so including self-assessments in grades is unnecessary and can compromise students' honesty. Always give students time to revise their work after assessing themselves …

(1997, 16).

With this practice, however, instructors must be clear that student assessments do not influence final grades; an instructor will not change a C to a B just because the student felt they deserved an A. For this reason, it may be useful to have students self-assess with rubrics that have descriptive modifiers rather than actual grades (e.g., instead of A, B, C, D, F, use the headings Exceptional, Well-Done, Fair, Needs Work, and Poor).

Peer Review

Another “off-label” but productive use of rubrics is to ask students to evaluate each other's work via peer review using the rubrics. (For this purpose, professors may want to use rubrics with descriptors rather than grades; in my experience, students did not want to assign grades to their peers.) In their study examining the use of rubrics for peer assessment, Hafner and Hafner concluded, “Peer learning and peer assessment are key pedagogical strategies to help students gain the knowledge that allows them to reflect on their own performance as well as that of their fellow students” (2003, 1526). Goodrich argues the benefits of self-assessment and peer review overlap: rubrics “help students become more thoughtful judges of the quality of their own and others' work. When rubrics are used to guide self- and peer assessment, students become increasingly able to spot and solve problems in their own and one another's work” (1997, 15).

As with self-assessment, instructors should be clear that peer reviews will not influence final grades, but peer reviews are provided to both help students learn to evaluate good work and to give the individual student a sense of peer opinion on their work. Should peer reviews be anonymous? As an instructor, I insist that students include their names on peer reviews, for two reasons. First, it forces students to be helpful rather than merely critical. Second, I explain to students that in the working world, they will be asked to provide feedback to their work colleagues on a regular basis, and this gives them good experience in doing so.

Creating Rubrics with Students

For small or upper-level courses, instructors may want to create grading rubrics with students. Having students help develop the rubric engages them, makes them think deeply about what is important in a particular assignment, and allows for organic construction rather than a bland, one-size-fits-all approach. In this way, a rubric “represents a consensus of the values of [a] community of writers” (Turley and Gallagher 2008, 92). Allowing students to participate in rubric construction gives them a sense of buy-in and ownership, making it more likely that they will take the assignment seriously. One study showed that inter-rater reliability increased when students helped to construct their own rubrics, likely because of this sense of buy-in. “The in-class exercise of building the rubric may have given the class … a heightened sense of ownership and a deeper understanding of the evaluative criteria and elements of the rubric than the other classes and may explain the higher reliability estimates for that year” (Hafner and Hafner 2003, 1521).

CONCLUSION: INCREASING THE QUALITY AND QUANTITY OF ANALYSIS WITH RUBRICS

Whereas professors may be aware that rubrics shorten grading time and improve grading consistency, they may not be aware that rubrics offer a powerful analytical punch. Given the demands for active learning in today's college classroom, rubrics allow instructors to focus on analytical quality while engaging students in a variety of assignments. Rubrics are useful not only in more traditional applications—for example, papers, and oral presentations—but also for more creative purposes. Both self- and peer assessment using rubrics engage students more deeply in assignments, allowing them to reflect on their own performance and their peers. Going one step further, instructors can help students connect to quality indicators and learning outcomes by inviting them to help construct the rubrics that be used to grade their own work. The main cost of rubric use is time, to ensure quality construction, but this time is saved many times over because rubrics streamline grading considerably. Finally, rubric use forces instructors to be clear about their own purposes for an assignment, and over time, instructors can become more attuned to the analytical possibilities in traditional “busywork” assignments. In total, rubrics offer serious analytical benefits in the political science classroom, with a minimum of cost—making them a tool that all professors should consider.