Does Research Collaboration Pay Off?
Collaborative (multi-investigator) research continues to spread from the natural and formal sciences to the social sciences, and it’s supposed to be a good thing. Among the much-ballyhooed virtues of collaboration are that it’s supposed to enable researchers to divide their labor in ways that expand the range of pertinent literatures within which any of them is familiar, to bring new methodological skills into play, and to produce synergies even when a team consists of researchers with very similar backgrounds and skill sets.
That all sounds great, but there’s distressingly little solid evidence that collaboration actually pays off. It’s pretty clear that researchers who work in groups tend to turn out more papers than lone wolves do, but are their papers really “better” in any meaningful sense? Collaboration has costs as well as benefits: because it often necessitates compromise, it may reduce risk-taking and innovation, leading to papers that are technically proficient but stale; and the outlooks, approaches, and preferences of the members of a multi-member research team may not meld well, making a patchwork product rather than an integrated whole.
In an article (gated; pre-publication version shown) in the current issue of PS: Political Science and Politics, I put one aspect of the alleged payoff of collaborative research – the publishability of the papers produced via collaboration – to the test, drawing on the record of submissions to the American Political Science Review during my six-year (fall 1991-summer 2007) stint as editor.
During that period, 7.5% of the papers that were submitted for review were ultimately accepted for publication. 55% of the submitted papers were single-authored; the rest were by teams ranging in size from two to nine. The overall acceptance rates of single- and multiple-authored papers were essentially identical (7.5% and 7.4%, respectively). That doesn’t tell the whole story, though, because acceptance rates varied among papers representing different parts of the discipline and different disciplines. With those differences taken into account, the following picture emerged:
(1) Overall, whether a paper was submitted by a single author or by two or more authors had no bearing on its chances of being accepted.
(2) But within that general pattern, a more specific pattern stood out. Single-authored papers did no worse than multiple-authored ones as long as at least one of the authors of the latter was a political scientist. Papers submitted by a single “outsider” fared much more poorly than those submitted by a single political scientist, multiple political scientists, or mixed teams with at least one political scientist. As I noted in the article, typically the single “outsider” was an economist, and their lack of success is consistent with the unflattering stereotype of economic imperialists “marching into neighboring disciplines without making much effort to acquaint themselves with those disciplines’ research literatures. Shortchanging contributions from outside of one’s own discipline might matter little when a paper is being considered by a journal in one’s own discipline. But seeking acceptance of one’s work without paying due heed to prior research in the journal’s own discipline … is likely to be self-defeating.”
This is a topic on which more research needs to be done, on a wide variety of journals and with a wide variety of measures of the “payoff” of collaboration.
Comments
But isn’t the appropriate study to compare the performance of the same authors submitting as single and as co-authors? After all, what one wants to know is if the likelihood of any individual producing work publishable in the APSR is greater when she works with a co-author than when she works independently.
Posted by: Thomas | July 16, 2009 09:29 AM
“drawing on the record of submissions to the American Political Science Review during my six-year (fall 1991-summer 2007) stint as editor.”
That must have been a long six years!
Posted by: Jim | July 16, 2009 09:42 AM
Thomas:
In your proposed study, there would be many of the same design weaknesses (or at least limits on inference) as in the one reported on above. To make it work really well, you would have to randomly determine whether the author was going to work alone or in a group on a paper, because all kinds of biases could creep into the decision to go it alone or seek out co-authors. I agree that a study like the one you imagine could turn up some interesting results, but so could a lot of very differently designed studies — which goes back to the last line of my post.
Posted by: Lee Sigelman | July 16, 2009 09:49 AM
In the natural and physical sciences collaborative research produces higher quality work; publishes in higher quality journals; results in researchers publishing more; and their works gets cited more frequently.
Very little work, as I’m sure you know, has looked at the social sciences. Collaboration has its costs and benefits…it all depends on what you are trying to get out of the collaboration. If you are mentoring a student, that collaboration will likely be a drag on your time compared to a collaboration with a colleague…though some colleagues can be drags as well.
Posted by: Ben C | July 16, 2009 10:10 AM
I would also pose an ancillary question to the one Lee looks at here: does who gets credit as an author vary by discipline as well?
I work in two fields, sociology and public health. From my experience, what is often given credit in public health as substantially contributing to authorship is a very different standard from what is considered as authorship contribution in sociology. For example, data managers/developers who create the data are often acknowledged in sociology while they are given authorship in public health. If this is systematically true, it is not really that researchers are gaining any benefit from co-authoring they are not already receiving in sole-authorship dominated fields like sociology (and, presumably, political science), but giving more credit or a higher level of credit to the work that would be done regardless of whether they are publishing as a team or individually.
This is obviously anecdotal since my experience is limited to two multidisciplinary research groups, one of mostly sociologists the other comprising mostly public health researchers, however, I think that it might lead to an interesting set of research questions asking whether different research paradigms or schemas in different disciplines lead to different types of research being done and what would happen if different schemas were used in different disciplines (e.g., would public health be more theoretical if there were more lone-wolves and would sociology be more applied with higher levels of co-authorship?.
Posted by: mike3550 | July 16, 2009 12:27 PM
Lee,
The numbers we’ve been crunching at American Politics Research on collaboration over the past three years tell a slightly different tale (http://www.bsos.umd.edu/gvpt/apr/coauthorship.html). Essentially, over the past three years, work with two authors received Revise and Resubmit decisions at a 14% higher rate than did those papers whose authors flew solo. The advantages of co-authorship seem to plateau there though—a paper with three or more authors had about the same odds of receiving an R&R as did one having two authors. We hadn’t thought about controlling for whether the coauthors are political scientists, however; that is an interesting wrinkle.
I’m glad you have kept up your analysis of APSR submission data even following your six-year (though it probably seemed like 16) term at the journal. Too few journals really analyze these data for insight into publication trends in our profession. Most of the published articles that attempt analysis of our journals focus solely on their output: published articles. These may miss biases—intentional or otherwise—that occur in the submission and peer review process. Analysis of the fuller universe of journal submissions provides a fuller picture of what is, and what is not, published in political science.
A big problem, however, is that these submission data are not widely collected, shared, nor distributed by all of our journals. I am working to collect submission data from many of the profession’s top journals for an article on transparency in journal submission data, as well as what the journals’ own data tell us about our reputational evaluations of journal quality. Hopefully I will be rewarded when I am ready to find a home for this article in keeping the lessons learned from the APR statistics on collaboration in mind: I am, of course, working with a coauthor. Fortunately, re the APSR data, she is also a political scientist.
Posted by: Steve Yoder | July 17, 2009 11:10 AM
Lee,
I’m glad that you are still running these data even following divvying up of your APSR editorial duties into nine equal parts. Our data at APR tell a somewhat different tale about the benefits of collaboration. Over the past three years, papers submitted to APR that had two coauthors were 14% more likely to receive a Revise and Resubmit than their solo-authored kin. This advantage seemed to plateau at two authors however, as papers with three or more authors were about as likely as those with two authors to receive an R & R (data are here: http://www.bsos.umd.edu/gvpt/apr/coauthorship.html).
Speaking more broadly, I’m glad that you are still digging deep into the APSR submission data. Few journal editors seem to relish self-evaluation, at least judging from the data that are readily accessible on journal web sites. Most of the articles published about our discipline’s journals (Garand and Giles, et al.) look at the journals output: published articles. To determine editorial bias—whether intentional or otherwise—as well as to paint a fuller picture of the work occurring in our discipline, we need to look at the wider universe of articles that are submitted to journals. It is well-nigh impossible for a journal to publish articles in a particular field or subfield, or with a specific methodological slant (touching on one of the main charges of alleged editorial bias during the Perestroika days), if such articles are not submitted. These data, again, are often not freely given by journal editors.
More transparency across the board on the type of articles each journal receives would also help authors better select where to send their work. If I knew that a particular journal received 100 articles on turnout, but only published 5, as opposed to receiving 100 on the judiciary and publishing 20, I would likely be less willing to submit my turnout article and more likely to submit my judiciary article there.
A coauthor and I have started to look at the data that journals collect on submissions, as opposed to what they publish. Most journal editors we have contacted have been very giving with their summary acceptance/R&R/rejection statistics, even if they do not trumpet these in annual reports or on their websites. It often seems the case that shifting editorships leads to loss of hard data, or the expectations of creating such hard data are not passed on to new journal offices or received from old journal offices. There seems very few set standards or guidelines for collecting or passing on journal submission data. In fact, in your experience what guidelines for running the journal did you receive from your predecessor upon starting your editorship?
Posted by: Steve Yoder | July 17, 2009 01:42 PM