While the Law and Society Review welcomes a range and diversity of law and society research, we remind potential authors to be sure to clearly discuss their data and methods in their manuscript to ensure that their research reviews well. The journal is methodologically plural, so all kinds of work are encouraged. But a clear statement of what one did, and how one did it, makes a manuscript easier for reviewers to read. It fits with practices in other journals, which is good for the field. It also speaks to current cross-disciplinary, cross-national concerns about data access and research transparency.
Publishers ask editors for their policies on transparency. The current editors of Law and Society Review take no position on the data access question for the purposes of the journal. Practices across disciplines and jurisdictions are too heterogeneous. This editors'note will sketch the debate and point toward resources for clarifying statements of data and methods.
The publications committee for the Law and Society Association is discussing changes in the publishing climate, including data access and research transparency pressures on journals, and reasons to be clear about data and methods. Many of the manuscripts submitted to LSR could be much clearer about data collection, and about how they analyze what they have. The data on which journal authors rely are not limited to one kind of information over another, nor do our authors share one type of analysis.
Political science and psychology, in particular, debate the value of accessible data and transparent research practices. Some journals require replication of data sets before they will publish. The American Journal of Political Science, for example, replicates all work before it can be published. Yet, a concern is that the pressure for data access and research transparency can color the work that gets done, leading to a fear that research that does not readily lend itself to replication cannot be published. Other threats to openness of research crop up. In economics, many writers rely on proprietary data—data purchased from a company—so the data are unavailable for replication. Research at Google and Facebook is itself proprietary, unavailable for use, for theory building or for replication. Thus, the demand for replication drives research, some kinds of research cannot get done, or the costs of making it possible for others to reuse data are too high. From the point of view of some computer scientists, the answer is to automate replication, but this would, of course, would drive the kind of research people could do.
Reference Freese and PetersonSociologists Jeremy Freese and David Peterson (2017) have recently argued for recognizing epistemic diversity within the replication debate, which translates into respecting diversity concerning whether making data accessible will serve scholarship well. If some research could never be reproduced, or never was imagined to be reproducible—say it's based on highly contextualized interpretations of field notes, or its creativity lies in the context in which data are interpreted—then reproducibility or even verification may not be the point of the work.
A partial way to address this sometimes very bitter debate is to rely on good practices explaining data and methods. Authors can ensure that someone who wanted to understand what an author did could understand it by following how documents or interviewees or websites were selected and interpreted. Online lives make it more possible to share interpreting: websites and photos and texts online can include links in reference lists, or active links for online publishing. A broad methodological literature allows people to clarify what they are doing and how. While we cannot address it all here, we offer a few references below to assist with writing one's work. The NSF report on qualitative methods cited below is available online.
Practices are always open for dispute. If you see specificity as an unhelpful practice that shuts out theorizing from the global South, for example, make the case. Tell readers how you did your research and why. On the LSR blog, Mihaela Serban critiqued the North American focus of too much sociolegal work. She also chided people for work that was so descriptive it did not invite others into caring about the work. These are important observations.
We look forward to your submissions: clearly theoretically motivated; data and methods clearly explained.
