Sarah Werner, Folger Shakespeare Library, Washington, DC
One of the surprises, for me, in running the open peer review for the special issue of Shakespeare Quarterly (SQ) I guest-edited on Shakespeare and performance was that all of the authors were willing to participate. Only one of the six who were being considered for publication had any hesitations, and those had to do with whether an essay that had been posted and then rejected would encounter difficulties in trying to place the piece elsewhere. After being reassured that SQ wouldn’t claim copyright over these submissions, and that a rejected essay could be taken down from the site, the author was happy to proceed. The other five authors seemed to have no qualms about having their work reviewed publicly, despite being reminded that essays in the open review were not necessarily going to be accepted for publication in the journal. (Indeed, one submission was rejected, and was subsequently withdrawn from the review site at the author’s request.) I had been anticipating a greater degree of jitters about the process, and I was happy that the authors were willing to try out this new format.
The other surprise, a less happy one, was how hard it was to convince reviewers to participate. I sent email announcements to list-servs and to the journal’s board, I tweeted, I Facebooked, I wrote to individual scholars and asked them to participate. And then I waited. And then I did it all again. Some people had specific qualms about participating having mainly to do with a sense of being on display. Those fears weren’t about their identity being revealed to the author, but about being visible to everyone else visiting the site and about being judged for whether their comments were smart enough. It’s helpful to point out what those fears were not: they were not worries about being held accountable for their comments, they were not disappointments about not being about to be snarky, they were not complaints that such judgments were best made privately. They were fears about being found wanting. They are fears that are probably familiar to many scholars.
But those specific fears were true only for some of the folks who were reluctant to review. There were many more who meant to review but who didn’t remember to, or who couldn’t find the time, or who just kept putting off getting it done. In retrospect, this ought not to have surprised me. We have all experienced something like it: when your to-do list is already too long, the items most likely to be dropped are those that someone else will pick up. Even when there is not an overwhelming to-do list, someone is less likely to act if they are part of a crowd that has been invited to act. This is why CPR classes teach us to specifically identify people to assist: don’t yell out, “someone call 911,” but rather make eye contact with one person and say, “you call 911.” When an invitation to review papers is sent out to hundreds of scholars, any one individual scholar is less likely to feel responsible for making sure those papers receive feedback. Even when individuals are singled out—“can you please review this paper on performances of Shakespeare on Mars?”—they seem less likely to follow through with comments than in the traditional peer review process. I do not have evidence as to why this is the case, but I do know that the number of requests it took to get an individual to comment was significantly higher than when we’ve sent out reviewer requests in our usual methods.
There was one exception to the general continual teeth-pulling it took to get comments on the submissions. As I’ve said, there were six submissions to the journal that were being vetted for publication, essays that covered a wide range of topics within the broad definition of “Shakespeare and performance.” But there was a seventh piece that was designated as a springboard for discussion, rather than a piece to be evaluated for publication. That piece was a review of an As You Like It production that both included details about the performance and raised questions about the nature of academic theatre reviewing. The review was put on the site and explicitly flagged as a starting point for discussion of both that particular production and the larger enterprise of reviewing, with the possibility of some comments appearing in print in the issue. I did solicit some direct responses, as I did for other pieces on the site, and generally treated it in the same manner as I had the submissions. But the rate of response was much higher: 52 comments were left on this review, more than on any other piece and nearly one-fifth of all the comments left on the site. The number of commenters responding was higher as well: there were 10 individuals commenting on the review, compared to a total of 30 individuals commenting throughout the site. Of that group of 10 commenters, four left five or more comments each. My point is that there was not only a greater concentration of comments and commenters on the springboard review, but that the comments took the form of a back-and-forth discussion between a group of readers. It’s a markedly different experience of engagement than is found elsewhere in the issue’s peer review.
So what made this section of the Shakespeare and performance cluster on MediaCommons so different from the others in the cluster? I have two hunches about that. One: it spoke to a smaller group of peers than the rest of the site. Shakespeare and performance studies encompasses a broad range of work, and that range was reflected in the submissions posted on the site. But academic theatre reviewing is practiced by a much smaller group of people, and thought about as a site of scholarly inquiry by an even smaller subset. The discussion of reviewing worked well because it’s a pressing concern to a group of individuals; we don’t all agree with each other, as is clear from the debate, but we do have a vested interest in thinking through these issues with our peers. My second hunch is that this section succeeded because reviewers weren’t being asked to pass judgment on the worthiness of the review but to engage with it as a topic of conversation. That is, instead of commenting on whether an aspect of it was right or wrong, new or not, commenters engaged with the ideas and questions being raised. Ideally, this wouldn’t be such a different activity that reviewing a submission. But I suspect it is.
Rather than seeing the negative side of this—the ways in which open peer review didn’t entirely succeed for my issue on Shakespeare and performance—I’m going to argue for the positive outtake from this. Open peer review offers the chance for small communities to engage with each other in open debate about the issues that define their field. For emergent scholarly communities, there is less of a need to control what gets published, and whether scholarship meets some standard of right or wrong, than there is for the issues of the field to be debated publicly and refined so as to draw new scholars into the community. Open peer review isn’t going to be one-size-fits-all any more than traditional, anonymous peer review. That’s one of the strengths of scholarly communication. It can, and should, take different forms to meet different needs.
The open peer review of the performance issue of Shakespeare Quarterly has been archived at http://mediacommons.futureofthebook.org/mcpress/shakespearequarterlyperformance/.
The print issue was published in Fall 2011; abstracts can be found at http://www.folger.edu/template.cfm?cid=3964.
My brief editor’s introduction can also be read online at http://sarahwerner.net/blog/index.php/2011/09/sq-issue-on-shakespeare-and-performance/, and the springboard conversation as it was edited for publication can also be found at http://sarahwerner.net/blog/index.php/rethinking-academic-reviewing/.
Building Community by Sarah Werner is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.