3 - Gathering and Analyzing Evidence

Summary

Introduction

SoTL involves the systematic investigation of a question we have about student learning and we look for answers in evidence generated by students. After framing a researchable question, we have to gather and analyze evidence. So this chapter examines some basic considerations of research design, such as whether, and how, to gather quantitative data, qualitative data, or both. It is likely that one or more of the types of evidence discussed in this chapter will be unfamiliar to mathematicians. Many of them were new to Curtis Bennett and me as well, when we began doing SoTL. In this chapter I write about these methods from our experience in learning to use them.

Triangulating Data

A SoTL researcher should develop a plan for systematically collecting multiple types of evidence. A diversity of evidence can help the researcher to form a convincing picture of student learning (Wiggins, 1998). This approach is called triangulation of the data. According to Webb, Campbell, Schwartz, and Sechrest (as cited in Shavelson & Towne, 2002, p. 64): “When a hypothesis can survive the confrontation of a series of complementary methods of testing, it contains a degree of validity unattainable by one tested within the more constricted framework of a single method.” In other words, claims or explanations supported by several types of evidence-for example, student work samples, interviews, and retention rates-are considered to be more accurate. This will be an asset if the work is submitted for publication in a peer-reviewed journal. We turn now to a discussion of the difficulties in approaching SoTL as standard educational research.

Challenges that Educational Research Design Presents for SoTL

Designing an educational research study can pose a number of challenges to faculty members interested in SoTL. Many of the same challenges confront mathematics education research at all levels. In response to a query from the NSF, and with its funding, the American Statistical Association held a series of workshops for statisticians and mathematics education researchers to discuss whether the statistics community could “offer any contributions to improving the quality of mathematics education research” (Working Group on Statistics in Mathematics Education Research, 2007, p. 1).