This post serves as an introduction to my interest in an alternative model of peer review; alt peer review.
First, I like peer review. Really! I can’t say how many times a paper I reviewed ends up being a paper I draw from in my work. That thought makes me think I should read and engage with more papers. Also, I like publishing papers. It’s a rewarding, if not (too) long process, and having one’s work be published in a journal—especially one that others or many in one’s professional field peruse or read—is an accomplishment.
But, I am also frustrated by peer review. While it is often great, leading one to change and improve one’s work substantially, it’s often … not. And, the publishing system relies on publishers that in some ways have policies and practices that are anti-reviewer (e.g., submissions guidelines that place tables and figures after the text of the paper), anti-reader (e.g., requiring readers to either have access through an [academic] library or to purchase articles), and anti-author (e.g., convoluted submission systems). Notably, authors and reviewers almost always do this work—often directly or indirectly funded by the public, or funded by students—entirely for free. I believe many editors do, too, and when they do receive funding, it’s typically a small amount to support a managing editor.
Awhile ago, a host of the Black Goat podcast (I think Simine Vazire shared a radically different idea for peer review, one in which individuals would review papers—pre-prints, published papers, and, I think, anything in between—through something like entries in a newsletter or posts on a blog. In this way, the distinction between something that is not peer-reviewed, like a pre-print, and something that is or was—a journal article—is blurred. This means that pre-prints can be reviewed, albeit in a less formal way. Also, articles that are already published can (still) be constructively critiqued. It also means that authors’ work could be the subject of a lot of attention if a prominent individual reviewed their work, regardless of the stage of the work, and that the review would become a part of the conversation around the work. There are a lot of possible problems that one could anticipate about this process (and many ways that it is less ideal than traditional peer review), but it struck me as a kind of review that is suited to doing research and science in an Internet-enabled way. I recognize that one downside of this approach is that the “review” can be trivial, not approaching the quality of even the median review from peer review as we know it.
There are other benefits. This would help me to read more work. Also, many (but certainly not all!) authors like to read about what people think of their work. Some fields have a more vibrant blogosphere; I think education research does not, yet. Finally, I like to try out new ways of doing research and writing. For instance, I wrote my dissertation in the open through GitHub not because I thought many people would read it (I think not many did!), but because I wanted to try to practice (and exhibit) this capability for future work (and, doing this led me to co-author an open book). Academics—even pre-tenure academics that are privileged because of their backgrounds and identity—have an incredible opportunity to pursue things they’re interested in and to do things in new ways. This is one area where I think academics can innovate.
I talked with my wife Katie (a librarian) about this idea (and, riffed on it with my podcast co-host Ryan), and Katie helped me to come up with (and Ryan helped me to settle on) the name What’s the point for what this alt peer review process may look like for me. So, I hope the authors of the paper that I’m reviewing don’t see the title of this post as pejorative! In fact, I admire the work, and, going forward, I plan to only read and write about papers that I’m interested in or have wanted to read. I don’t really have a format yet, but I’ll try to figure that out here.