The flipped classroom is a new approach to higher education that relies heavily on active-learning techniques. Traditional pedagogies—such as reading textbooks, attending lectures, or a combination of both—are widely criticized for their lack of efficacy in achieving a host of learning outcomes, from understanding basic concepts to critical thinking (Barr and Tagg Reference Barr and Tagg1995). The flipped classroom, by contrast, makes the transmission of foundational knowledge part of students’ self-directed learning at home while reserving classroom time for higher-order learning activities. Whereas the majority of flipped classrooms have been developed in STEM programs, they are increasingly used in the social sciences as well (Roehling Reference Roehling2017), including political and policy sciences (Jenkins Reference Jenkins2015; Touchton Reference Touchton2015; Whitman Cobb 2016). Although flipped classrooms do not require blended learning, many incorporate online teaching tools. This article describes our experiences in implementing a blended flipped classroom in the redesign of an existing course on research methods in the Institute of Public Administration at Leiden University.
The new approach combines a flipped classroom with blended learning, two course designs that have become more common in political science teaching (Salter Reference Salter2013; Whitman Cobb 2016). The course combines online and face-to-face learning, whereas the traditional division of labor between lecture time and homework is reversed: students acquire basic knowledge at home in an online environment and use their time in class to apply this knowledge and practice their skills. The blended flipped classroom does not simply replicate the traditional lecture online. Instead, it relies on active-learning techniques using exercises and assignments that allow students to learn by doing, including enhanced lectures and mini-games (Asal et al. Reference Asal, Jahanbani, Lee and Ren2018; Bromley Reference Bromley2013). It also incorporates social learning in the course design, expecting students to learn from others through small-group work and peer review (Bromley Reference Bromley2013).
Content is not offered sequentially to students; instead, they choose the methods in which they want to specialize from several modules. At the graduate level, most students no longer need a broad survey course to carry out their research projects. Giving students the freedom of choice, particularly for high-anxiety courses such as research methods, has been found to have a positive effect on performance and motivation (Hardway and Stroud Reference Hardway and Stroud2014). It also allows the incorporation of a host of different research methodologies—both qualitative and quantitative—to do justice to the methodological pluralism in our discipline (Mahoney and Goertz Reference Mahoney and Goertz2006).
This article outlines what the blended flipped classroom approach to teaching social science research methods consists of and how we implemented it in our course. This course is taught twice a year in the master’s program in public administration at Leiden University. It is a mandatory seven-week course in the curriculum, taught between a course on research design and the independent thesis project. Enrollment fluctuates between 50 and 80 students per course edition. Whereas the online environment is immediately available when the course begins, the offline workshops are taught at different times throughout the seven-week period. These reported findings are derived from three editions of the course taught in the academic years 2016–2017 and 2017–2018.
RESEARCH METHODS, BLENDED AND FLIPPED
The great challenge of a blended flipped classroom in research methods is to provide sufficient “structured flexibility” to allow students to work at their own pace without leaving them completely to their own devices (O’Flaherty et al. Reference O’Flaherty, Phillips, Karanicolas, Snelling and Winning2015). This course design creates structure in two ways. First, online modules are divided into three distinct categories: data collection, data analysis, and academic skills. Whereas the academic-skills modules are mandatory for everyone, students are free to choose which module(s) from the data-collection and data-analysis sections they take. Second, clear connections are made between the online and offline components of the course: what students learn in the online modules is immediately applied in the offline workshops, which then is integrated into the concluding assignment for each module. Each edition of the course has four distinct phases (figure 1).
Figure 1 Course Design of the Blended Flipped Classroom
Phase 1 is a face-to-face meeting in which instructors and students meet one another for the first time. With much of the teaching taking place online, it is important to establish from the onset that this is not an only-online course such as a massive online open course or a small personal online course. After the introductory meeting, students have six weeks to select and follow several online modules, attend workshops, and write their assignments (i.e., phase 2). The online-course environment within the online platform currently consists of 11 modules organized around the most common methods for data collection and analysis used in the political and policy sciences (see figure 1). Students must take a minimum of seven modules and can volunteer to do more. A module typically consists of three to four knowledge clips (a maximum of 7 minutes), one or two readings, a quiz, and an assignment.
These face-to-face components of the course are important: during the workshops, students engage in group-based learning activities and receive direct feedback from an instructor.
In phase 3 of the course, students participate in offline workshops related to the online modules. They attend workshops only for the mandatory and voluntary modules that they followed online: typically, three or four because not all modules (e.g., data management) currently have an associated workshop. Workshops are two- or three-hour meetings that occur multiple times throughout the seven-week period of the course. To gain entry to a workshop, students must submit their draft assignment as instructed in the online part of the course. During the workshop, they use their assignment during in-class exercises, and strategies for successful completion of the assignment are discussed. Examples include mock interviews in a “fishbowl” format, an online scavenger hunt to locate quality primary sources, and—taking inspiration from Collier (Reference Collier2011)—a criminal investigation to apply process-tracing (see supplementary material). These face-to-face components of the course are important: during the workshops, students engage in group-based learning activities and receive direct feedback from an instructor. To give students flexibility and to maintain small class sizes, all workshops are offered at least twice and in different weeks.
Phase 4 of the course is assessment: six assignments (30% of the total grade, 5% each) and a written exam (70%). The graded assignments are revised versions of those that students submitted to gain entry to the workshops—when students take the workshop, many realize that they made mistakes. They then are given the opportunity to revise and resubmit their earlier work, before the assignments are graded. After a final face-to-face meeting with all groups of students and instructors, they take a written exam. The exam mirrors the course structure with three different sections: academic skills, data collection, and data analysis. Students must answer one question for each of the mandatory academic-skills modules and choose one question from the data-collection and data-analysis sections. Those who took more than one data-collection or data-analysis module are allowed to choose which question from each section they want to answer. For instance, students are asked to edit a short text that contains obvious flaws in writing, assess the quality of questions in an interview transcript, or interpret a statistical output.
WHICH RESOURCES ARE NEEDED?
The blended flipped classroom approach to teaching research methods relies heavily on digital technologies to deliver course content. Although it is argued that such technologies are particularly beneficial to tech-savvy millennials (Roehl, Reddy, and Shannon Reference Roehl, Reddy and Shannon2013), they do require a substantial investment in time and resources. Therefore, we decided to engage as much as possible with technologies already available to us. First, we used the existing online-course environment used by our university (i.e., “Blackboard”) as the platform for the online modules. Second, we restricted our time in the film studio by recording only the introductory videos for each module. For the substance of each module, we produced our own knowledge clips, using Prezi—a high-quality microphone for voiceovers and screen-capture software. This approach made us less dependent on outside expertise and material and also made it possible to produce content with minimal material requirements. Other scholars have produced and published excellent knowledge clips online; instead of making a similar clip, we linked our modules to those videos. Third, we used the interactive features of Blackboard to set up platforms where students could sign up for workshops, ask questions about the course material, and report problems with the technology.
Nevertheless, the project would not have been possible without substantial institutional support provided by the university. The blended flipped classroom was developed with funds received from the Leiden University’s ICT & Education program, the Faculty of Governance and Global Affairs, and the Netherlands Ministry of Education. Financial support included a grant that allowed instructors to hire teaching assistants, reduce their overall teaching load, and purchase recording material and software. Material support included the availability of an on-campus studio to film the introductory videos, supervised by a professional videographer. Instructional designers, employed by the university’s Online Learning Lab, helped with course design.
Neither the online modules nor the face-to-face workshops required more material resources from students than a traditional classroom setting. They already had access to the existing course-management system and they could complete the online modules on any university computer if they did not have their own computer. Although readings were mandatory for all modules, the course did not require an expensive textbook; the modules were linked to the university library’s catalogue, where a digital copy of publications could be accessed. However, the course required a substantial time investment—admittedly more than the non-flipped version of the course that was taught previously. For that reason, each module component provided an estimate of how much time was needed to complete it. When we received complaints that some estimates did not reflect the actual time a student spent on a component, we made adjustments accordingly.
UNEXPECTED OUTCOMES AND LESSONS LEARNED
Although flipped classrooms have become a veritable fad in recent years, from K–12 to higher education, evidence of positive effects on learning outcomes is still inconclusive. Many studies show a positive effect on student performance, but these tend to be restricted to (predominantly STEM) courses with high levels of foundational-knowledge transmission. Studies also often suffer from a lack of comparative data on student performance in flipped and traditional classrooms. Instead, they rely on self-reporting by students, which is shaped by considerations other than the pure “learning value” of courses. Research also indicates that the observed benefits of the flipped classroom actually may be caused by the use of active-learning techniques rather than the inverted classroom. For instance, the difference in performance between flipped and traditional classrooms disappears when both use active-learning techniques (DeLozier and Rhodes Reference DeLozier and Rhodes2017; O’Flaherty et al. Reference O’Flaherty, Phillips, Karanicolas, Snelling and Winning2015).
Similar caveats apply here: we cannot provide a systematic comparative assessment of the added value of the flipped classroom in comparison with a “traditional” methods course. The previous non-flipped edition of the course did not use such active-learning techniques. Moreover, we could cover a broader range of topics in the flipped-classroom version of the course, which makes the “flipped” and “non-flipped” versions not directly comparable. However, a short qualitative assessment of students’ written feedback provides insight into what they perceived as the main benefits and problems of the flipped classroom. For instance, the first iteration of the flipped classroom made some students feel “like a lab rat being experimented on.” However, feedback addressing the format was generally positive, highlighting the advantages of its flexibility—for instance, the possibility to “work at my own pace and in my own time” and “pick the specific research instruments you needed.” This flexible approach, however, also raised “problems with self-discipline”; because there was no weekly class, “I tended to push back doing the modules.” Critical comments focused on the course workload, which indeed was expanded with the introduction of the flipped classroom.
Moreover, students showed a greater mastery of research-methods skills in the assignments and on the final exam than in the previous course format.
We also observed—in line with the literature—that to be successful, flexible pedagogies require “flexible students” (Wanner and Palmer Reference Wanner and Palmer2015). Hence, we encountered resistance to the format of the blended flipped classroom. Particularly in the first edition of the course, some students found the instructors to be too absent from their daily learning experiences. One student commented in the course evaluation as follows: “The teachers are only using this format to reduce their own workload. Why can’t they just give a traditional lecture instead of letting us do all the work?” The introductory meeting for each new edition of the course has been changed to begin with an extensive explanation of the rationale for the blended flipped classroom. It offers students flexibility and specialization but also requires active involvement in their own learning experience. It is a message that is repeated continuously, from the homepage of the online-course environment to the various offline workshops that are offered.
Mindful of the limitations of comparison, the average scores of student evaluations for the first iteration of the flipped classroom were not surprisingly lower than the last iteration of the non-flipped version. Course evaluations are administered after the final exam and consist of 15 statements covering a range of topics from perceptions of workload to perceptions of the instructor’s performance. Students must indicate their agreement with the statements on a five-point Likert scale. For “The quality of the course was [1=poor, 2=insufficient, 3=sufficient, 4=good, 5=excellent],” the non-flipped course scored 3.8 (N=26) and the first edition of the flipped classroom scored 2.7 (N=74). However, these scores came closer to old levels in subsequent course editions, as issues related to the format and integration with other courses were resolved. This highlights the idea that “flipping” the classroom is an investment over time, which requires fine-tuning and improvement throughout a period of trial and error.
Moreover, students showed a greater mastery of research-methods skills in the assignments and on the final exam than in the previous course format. By the time students take the final exam, they already have practice with the course material, which often pays off: the final exam has a high passing rate, despite that students find it more demanding. So far, it has fluctuated between 84% and 94% each course edition (pass rates for the course tend to be lower because students often need to revise and resubmit module assignments). Furthermore, colleagues involved in thesis supervision report an across-the-board improvement in research skills that students display in their individual projects. This has benefited not only the students but also the thesis supervisors who spend less time correcting students’ work. Unfortunately, we do not have data available to demonstrate the effect of the curriculum on courses other than our own.
A final unexpected outcome of the flipped-classroom project was the new connection it forged between the research-methods course and the education provided by faculty in the department. To some extent, this connection already was embedded in the course design: the flipped classroom includes a podcast channel on which our colleagues discuss their use of particular methods in their research. They also are helping to design new modules that teach the methods in which they specialize. With all knowledge clips created under a Creative Commons Share Alike license, moreover, colleagues can use the curriculum in their own courses or in thesis supervision.
Outside of the course, the curriculum has served two purposes. First, colleagues teaching other courses have assigned parts of our curriculum if they believed students needed a quick refresher on a particular topic. This has reduced their workload, particularly in the context of thesis supervision. Second, the course has set explicit standards in academic writing, research ethics, and data management throughout the graduate program, where these were previously implicit. In this respect, the blended flipped classroom on research methods has acted as a common good in the teaching profile of our institution.
CONCLUSION
The transition from an offline to an online course can be daunting in many respects. On the one hand, instructors may be reluctant to engage the time and other resources necessary to develop new teaching material online, especially in a context in which research and publications seem to be more important for tenure and career advancement. Moreover, there is often the idea that recorded material must be perfectly polished. In our practice, however, we found that it is possible to develop effective online teaching materials with a relatively small investment in technical resources and without depending too much on outside expertise. On the other hand, students may be reluctant to engage with materials that are only online; our experience shows that there is relatively little support for the complete suppression of the offline component of the course we taught. In this context, the flipped classroom in research methods constituted a good compromise between the flexibility allowed by online courses and the social aspect of face-to-face interactions in class that is still valued by students.
What is not often mentioned, however, is the collective good created through the production of online teaching material. It is used not only within the remit of one particular course; it also can be shared within a department and beyond, thereby contributing to better integration and higher-quality standards across education programs. The goal now is to use colleagues’ expertise on particular research methodologies to complement that of instructors. Funding has been obtained for other scholars to develop new online modules on methods not yet covered in the course. Thus, the range of methods offered can be expanded quite extensively while the course begins to rely less heavily on instructors—which furthers the continuity of the project.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049096519000581
ACKNOWLEDGMENTS
We gratefully acknowledge the financial support of Leiden University’s ICT & Education program, the Faculty of Governance and Global Affairs, and the Netherlands Ministry of Education. We thank Annemieke van den Bijllaardt, Thomas Ginn, Joasia van Kooten, and Marja Verstelle for their indispensable help in creating and implementing the flipped classroom.
