The role of conference publications in CS

Abstract

A bibliometric view of the publishing frequency and impact of conference proceedings compared to archival journal publication.

The role of conference publications in computer science is controversial. Conferences have the undeniable advantages of providing fast and regular publication of papers and of bringing researchers together by offering the opportunity to present and discuss the paper with peers. These peculiar features of conferences are particularly important because computer science is a relatively young and fast-evolving discipline. The fundamental role of conferences in computer science is underlined with strength in the best-practices memo for evaluating computer scientists and engineers for promotion and tenure published in 1999 by the U.S. Computing Research Association (CRA) and, more recently, in a study of the Informatics Europe, whose preliminary results are summarized in Choppy et al.

Recently, Communications published a series of thought-provoking Viewpoint columns and letters that swim against the tide. These contributions highlight many flaws of the conference system, in particular when compared to archival journals, and also suggest a game-based solution to scale the academic publication process to Internet scale. Some of the mentioned flaws are: short time for referees to review the papers, limited number of pages for publication, limited time for authors to polish the paper after receiving comments from reviewers, and overload of the best researchers as reviewers in conference program committees. The result is a deadline-driven publication system, in which "we submit a paper when we reach an appropriate conference deadline instead of when the research has been properly fleshed out," that "encourages and rewards production of publishing quarks---units of intellectual endeavor that can be generated, summarized, and reviewed in a calendar year" (interestingly, the author of the latter claim is CRA Board Chair Dan Reed). Furthermore, the current conference system "leads to an emphasis on safe papers (incremental and technical) versus those that explore new models and research directions outside the established core areas of the conferences." "And arguably it is the more innovative papers that suffer, because they are time consuming to read and understand, so they are the most likely to be either completely misunderstood or underappreciated by an increasingly error-prone process." Are we driving on the wrong side of the publication road? The question is raised by Moshe Vardi in a May 2009 Communications editor's letter.

This article gives an alternative view on this hot issue: the bibliometric perspective. Bibliometrics has become a standard tool of science policy and research management in the last decades. In particular, academic institutions increasingly rely on bibliometric analysis for making decisions regarding hiring, promotion, tenure, and funding of scholars. I investigate the frequency and impact of conference publications in computer science, comparing with journal articles. I stratify the set of computer science publications by author, topic, and nation; in particular, I analyze publications of the most prolific, most popular, and most prestigious scholars in computer science.

References

1. Birman, K. and Schneider, F. B. Program committee overload in systems. Commun. ACM 52, 5 (May 2009), 34--37.
2. Choppy, C. et al. Research evaluation for computer science. Commun. ACM 52, 4 (Apr. 2009), 31--34.
3. Fortnow, L. Time for computer science to grow up. Commun. ACM 52, 8 (Aug. 2009), 33--35.
4. Franceschet, M. The difference between popularity and prestige in the sciences and in the social sciences: A bibliometric analysis. Journal of Informetrics 4, 1 (2010), 55--63. DOI: 10.1016/j.joi.2009.08.001.
5. Hirsch, J. E. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of USA 102, 46 (2005), 16569--16572.
6. King, D. A. The scientific impact of nations. Nature, 430 (2004), 311--316.
7. McKeown, N., Crowcroft, J., and Keshav, S. Scaling the academic publication process to Internet scale. Commun. ACM 52, 1 (Jan. 2009), 27--30.
8. Reed, R. Publishing quarks: Considering our culture. Computing Research News 21, 2 (Feb. 2009).
9. Vardi, M. Conferences vs. journals in computing research. Commun. ACM 52, 5 (May 2009), 5.

Index Terms

• Applied computing

  • Computers in other domains

    • Digital libraries and archives

• Information systems

  • Information retrieval

  • Information storage systems

  • Information systems applications

    • Digital libraries and archives