ACM Digital Library home
ACM home
Advanced Search
article

Google Scholar citations and Google Web-URL citations: A multi-discipline exploratory analysis

Authors Info & Claims
Journal of the American Society for Information Science and TechnologyVolume 58Issue 7May 2007 pp 1055–1065
Online:01 May 2007Publication History
  • 41citation
  • 0
  • Downloads
Metrics
Total Citations41
Total Downloads0
Last 12 Months0
Last 6 weeks0

Abstract

We use a new data gathering method, “Web/URL citation,” Web/URL and Google Scholar to compare traditional and Web-based citation patterns across multiple disciplines (biology, chemistry, physics, computing, sociology, economics, psychology, and education) based upon a sample of 1,650 articles from 108 open access (OA) journals published in 2001. A Web/URL citation of an online journal article is a Web mention of its title, URL, or both. For each discipline, except psychology, we found significant correlations between Thomson Scientific (formerly Thomson ISI, here: ISI) citations and both Google Scholar and Google Web/URL citations. Google Scholar citations correlated more highly with ISI citations than did Google Web/URL citations, indicating that the Web/URL method measures a broader type of citation phenomenon. Google Scholar citations were more numerous than ISI citations in computer science and the four social science disciplines, suggesting that Google Scholar is more comprehensive for social sciences and perhaps also when conference articles are valued and published online. We also found large disciplinary differences in the percentage overlap between ISI and Google Scholar citation sources. Finally, although we found many significant trends, there were also numerous exceptions, suggesting that replacing traditional citation sources with the Web or Google Scholar for research impact calculations would be problematic. © 2007 Wiley Periodicals, Inc.

References

  1. About Google Scholar. (2005). Retrieved December 12, 2005, from http://scholar.google.com/scholar/about.htmlGoogle ScholarGoogle Scholar
  2. Almind, T. C., & Ingwersen, P. (1997). Informetric analyses on the World Wide Web; Methodological approaches to "Webometrics." Journal of Documentation, 53(4), 404-426.Google ScholarGoogle ScholarCross RefCross Ref
  3. Antelman, K. (2004). Do Open-Access articles have a greater research impact? College & Research Libraries, 65(5), 372-382.Google ScholarGoogle ScholarCross RefCross Ref
  4. Bar-Ilan, J. (1999). Search engine results over time--A case study on search engine stability. Cybermetrics 2/3. Retrieved January 26, 2006. from http://www.cindoc.csic.es/cybermetrics/articles/v2i1p1.htmlGoogle ScholarGoogle Scholar
  5. Bar-Ilan, J. (2004). The use of Web search engines in information science research. Annual Review of Information Science and Technology, 38, 231-288.Google ScholarGoogle Scholar
  6. Bar-Ilan, J. (2005). Expectations versus reality-Search engine features needed for Web research. Cybermetrics, 9(1). Retrieved July 21, 2006, from http://www.cindoc.csic.es/cybermetrics/articles/v9i1p2.htmlGoogle ScholarGoogle Scholar
  7. Bauer, K., & Bakkalbasi, N. (2005). An examination of citation counts in a new scholarly communication environment. D-Lib Magazine, 11(9). Retrieved December 23, 2005, from http://www.dlib.org/dlib/september05/ bauer/O9bauer.htmlGoogle ScholarGoogle Scholar
  8. Belew, R. K. (2005). Scientific impact quantity and quality: Analysis of two sources of bibliographic data. Retrieved April 28, 2006 from http://arxiv.org/ abs/cs.IR/0504036Google ScholarGoogle Scholar
  9. Björneborn, L., & Ingwersen, P. (2001). Perspectives of Webometrics. Scientometrics, 50(1), 65-82.Google ScholarGoogle Scholar
  10. Borgman, C., & Furner, J. (2002). Scholarly Communication and bibliometrics. Annual Review of Information Science and Technology, 36, pp. 3-72. Medford, NJ: Information Today Inc.Google ScholarGoogle Scholar
  11. Brin, S., & Page, L. (1998). The anatomy of a large scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30(1-7), 107-117. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Brody, T., Stamerjohanns, H., Vallières, F., Harnad, S., Gingras, Y., & Oppenheim, C. (2004). The effect of open access on citation impact. Retrieved November 13, 2001, from http://www.ecs.soton.ac.uk/~harnad/ Temp/OA-TAadvantage.pdfGoogle ScholarGoogle Scholar
  13. Egghe, L. (2000). New informetric aspects of the Internet: Some reflections-- many problems. Journal of Information Science, 26(5), 329-335.Google ScholarGoogle ScholarCross RefCross Ref
  14. Friend, F (2006) Google Scholar: Potentially good for users of academic information. The Journal of Electronic Publishing. Retrieved April 28, 2006, from http://hdl.handle.net/2027/spo.3336451.0009.105Google ScholarGoogle Scholar
  15. Fry, J. (2004). The cultural shaping of ICTs within academic fields: Corpus-based linguistics as a case study. Literary and Linguistic Computing, 19(3), 303-319.Google ScholarGoogle ScholarCross RefCross Ref
  16. Fry, J., & Talja, S. (2004). The cultural shaping of scholarly communication: Explaining e-journal use within and across academic fields. In Proceedings of the 67th ASIST Annual Meeting 2004 (Vol. 41. pp. 20-30). Medford, NJ: Information Today.Google ScholarGoogle Scholar
  17. Glänzel, W. (2003, February). On some on some principle differences between citations and sitation links. A methodological and mathematical approach. Nerdi lecture delivered at NIWI, KNAW, Amsterdam. (Updated version of a paper presented at the 6th Nordic Workshop on Bibliometrics, Stockholm, October 4-5, 2001.)Google ScholarGoogle Scholar
  18. Goodrum, A. A., McCain, K. W., Lawrence, S., & Giles, C. L. (2001). Scholarly publishing in the Internet age: A citation analysis of computer science literature. Information Processing & Management, 37(5), 661-676.Google ScholarGoogle Scholar
  19. Harnad, S. (1990). Scholarly skywriting and the prepublication continuum of scientific inquiry. Psychological Science 1, 342-343. Retrieved November, 12, 2004, from http://www.cogsci.soton.ac.uk/~harnad/ Articles/Harnad/harnad90.skywriting.htmlGoogle ScholarGoogle ScholarCross RefCross Ref
  20. Harnad, S. (1991). Post-Gutenberg galaxy: The fourth revolution in the means of production of knowledge. Public-Access Computer Systems Review, 2(1), 39-53. Retrieved November 12, 2004, from http://www.cogsci. soton.ac.uk/~harnad/Articles/Harnad/harnad91.postgutenberg.html/Google ScholarGoogle Scholar
  21. Harnad, S., (1999). The future of scholarly skywriting, in the sky: Visions of the information future. Retrieved November 12, 2004, from http:// cogprints.org/1698/00/harnad99.aslib.htmlGoogle ScholarGoogle Scholar
  22. Harnad, S., & Brody, T. (2004). Comparing the impact of open access (oa) vs. non-oa articles in the same journals. D-Lib Magazine, 10(6). Retrieved May 2, 2006, from http://www.dlib.org/dlib/june04/harnad/06harnad.htmlGoogle ScholarGoogle Scholar
  23. Harnard, S., & Carr, L. (2000). Integrating, navigating, and analysing open eprint archives through open citation linking (the OpCit project). Current Science, 79(5), 629-638.Google ScholarGoogle Scholar
  24. Harter, S. P. (1996). The impact of electronic journals on scholarly communication: A citation analysis. The Public-Access Computer Systems Review, 7. Retrieved November 13, 2001, from http://info.lib.uh.edu/ pr/v7/n5/hart7n5.htmlGoogle ScholarGoogle Scholar
  25. Harter, S., & Ford, C. (2000). Web-based analysis of E-journal impact: Approaches, problems, and issues. Journal of the American Society for Information Science, 51(13), 1159-76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Herring, S. D. (2002). Use of electronic resources in scholarly electronic journals: A citation analysis. College and Research Libraries, 63(4), 334-340.Google ScholarGoogle ScholarCross RefCross Ref
  27. Ingwersen, p. (1998). The calculation of Web impact factors. Journal of Documentation, 54(2), 236-243. ISI press release essay on the impact of open access journals: A citation study from Thomson Scientific. Retrieved November 13, 2004, from http://www.ISInet.com/oajGoogle ScholarGoogle Scholar
  28. Jacso, P. (2004). Google Scholar Beta. Péter's Digital Reference Shelf. Retrieved Jan 10, 2006, from http://snipurl.com/dwcoGoogle ScholarGoogle Scholar
  29. Jacso, P. (2005a). Google Scholar: The pros and the cons. Online Information Review, 29(2), 208-214.Google ScholarGoogle ScholarCross RefCross Ref
  30. Jacso, P. (2005b). As we may search: Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Science, 89(9), 1537-1547. Retrieved April 28, 2006, from http://www.ias.ac.in/currsci/nov102005/1537.pdfGoogle ScholarGoogle Scholar
  31. Kim, H. J. (2000). Motivations for hyperlinking in scholarly electronic articles: A qualitative study. Journal of the American Society for Information Science, 51(10), 887-899. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Kling, R., & Callahan, E. (2003). Electronic journals, the internet, and scholarly publishing. Annual Review of Information Science and Technology, 37, 127-177.Google ScholarGoogle ScholarCross RefCross Ref
  33. Kling, R., & McKim, G. (1999). Scholarly communication and the continuum of electronic publishing. Journal of American Society for Information Science, 50(10), 890-906. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Kousha, K., & Thelwall, M. (2006). Motivations for URL citations to open access LIS library and information science articles: Exploring characteristics of sources of Web citation. Scientometrics, 68(3), 501-517.Google ScholarGoogle ScholarCross RefCross Ref
  35. Kurtz, M. J. (2004). Restrictive access policies cut readership of electronic research journal articles by a factor of two. Harvard-Smithsonian Centre for Astrophysics, Cambridge, MA. Retrieved November 13, 2001, from http://opcit.eprints.org/feb19oa/kurtz.pdfGoogle ScholarGoogle Scholar
  36. Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., & Murray, S. S. (2005). Worldwide use and impact of the NASA Astrophysics Data System digital library. Journal of the American Society for Information Science & Technology, 56(1), 36-45. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Lawrence, S. (2001). Free online availability substantially increases a article's impact. Nature, 411, 521. Retrieved November 13, 2001, from http://www.nature.com/nature/debates/e-access/Articles/lawrence.htmlGoogle ScholarGoogle ScholarCross RefCross Ref
  38. Lawrence, S., & Giles, C. L. (1999). Accessibility of information on the Web. Nature, 400, 107-109. Google ScholarGoogle Scholar
  39. Mayr, P., & Tosques, F. (2005). Google Web APIs: An instrument for Webometric analyses? Retrieved January 20, 2006, from http://www.ib. hu-berlin.de/%7Emayr/arbeiten/ISSI2005_Mayr_Toques.pdfGoogle ScholarGoogle Scholar
  40. Mettrop, W., & Nieuwenhuysen, P. (2001). Internet search engines-fluctuations in document accessibility. Journal of Documentation, 57(5), 623-651.Google ScholarGoogle ScholarCross RefCross Ref
  41. Moed, H. F. (2005). Citation analysis in research evaluation. New York: Springer. Google ScholarGoogle Scholar
  42. Notess, G. (2003). Google inconsistencies. Search Engine Showdown. Retrieved July 21, 2006, from http://www.searchengineshowdown.com/ features/google/inconsistent.shtmlGoogle ScholarGoogle Scholar
  43. Notess, G. R. (2005). Scholarly Web searching: Google Scholar and Scirus. Online, 29(4).Google ScholarGoogle Scholar
  44. Oppenheim, C. (2000). Do patent citations count? In B. Cronin & H. B. Atkins (Eds.), The Web of knowledge: A festschrift in honor of Eugene Garfield (pp. 405-432). Metford, NJ. Information Today Inc ASS Monograph Series.Google ScholarGoogle Scholar
  45. Pauly, D., & Stergiou, K. (2005). Equivalence of results from two citation Thomson ISI's Citation Index and Google's Scholar service. Ethics in Science and Environmental Policies. December 22, 33-35.Google ScholarGoogle ScholarCross RefCross Ref
  46. Rousseau, R. (1997). Sitations: An exploratory study. Cybermetrics, 1(1). Retrieved November 14, 2001, from http://www.cindoc.csic.es/ cybermetrics/articles/v2i1p2.htmlGoogle ScholarGoogle Scholar
  47. Shadbolt, N., Brody, T., Carr, L. & Harnad, S. (2006). The open research Web: A preview of the optimal and the inevitable. In N. Jacobs (Ed.), Open access: Key strategic, technical and economic aspects. Chandos. Retrieved May 3, 2006, from http://cogprints.org/4841/02/shad-bch.pdfGoogle ScholarGoogle Scholar
  48. Shin, E.-J. (2003). Do Impact Factors change with a change of medium? A comparison of impact factors when publication is by article and through parallel publishing. Journal of Information Science, 29(6), 527-533.Google ScholarGoogle ScholarCross RefCross Ref
  49. Smith, A. G. (1999). A tale of two Web spaces: Comparing sites using Web impact factors. Journal of Documentation, 55(5), 577-592.Google ScholarGoogle Scholar
  50. Smith, A. G. (2004). Web links as analogues of citations. Information Research, 9(4). Retrieved March 20, 2005, from http://informationr.net/ ir/9-4/article188.htmlGoogle ScholarGoogle Scholar
  51. Swan, A., & Brown, S. (2004). Report of the JISC/OSI open access journal authors survey, 1-76. Retrieved April 20, 2006, from http://www. jisc.ac.uk/uploaded_documents/JISCOAreport1.pdfGoogle ScholarGoogle Scholar
  52. Swan, A., & Brown, S. (2005). Open access self-archiving: An author study, 1-97. Retrieved April 20, 2006, from http://eprints.ecs.soton.ac.uk/10999/ O1/jisc2.pdfGoogle ScholarGoogle Scholar
  53. Thelwall, M. (2004). Link analysis: An information science approach. San Diego: Academic Press.Google ScholarGoogle Scholar
  54. Thelwall, M. (2006). Interpreting social science link analysis research: A theoretical framework. Journal of the American Society for Information Science and Technology. 57(1), 60-68. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Thelwall, M., Vaughan, L., & Björneborn, L. (2005). Webometrics. Annual Review of Information Science and Technology, 39, 81-135. Medford, NJ: Information Today Inc.Google ScholarGoogle ScholarCross RefCross Ref
  56. Van Impe, S., & Rousseau, R. (2006). Web-to-print citations and the humanities. Information--Wissenschaft und Praxis, 57(8), 422-426.Google ScholarGoogle Scholar
  57. Vaughan, L. (2004). New measurements for search engine evaluation proposed and tested. Information Processing & Management, 40(4), 677-691. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Vaughan, L., & Hysen, K. (2002). Relationship between links to journal Web sites and Impact Factors. Aslib Proceedings, New Information Perspectives, 54(6), 356-361.Google ScholarGoogle Scholar
  59. Vaughan, L., & Shaw, D. (2003). Bibliographic and Web citations: What is the difference? Journal of the American Society for Information Science and Technology, 54(14), 1313-1324. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Vaughan, L., & Shaw, D. (2005). Web citation data for impact assessment: A comparison of four science disciplines. Journal of the American Society for Information Science and Technology, 56(10), 1075-1087. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Vaughan, L., & Thelwall, M. (2003). Scholarly use of the Web: What are the key inducers of links to journal Web sites? Journal of the American Society for Information Science and Technology, 54(1), 29-38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Whitley, R. (2000). The intellectual and social organization of the sciences (2nd ed.). Oxford: Oxford University Press.Google ScholarGoogle Scholar
  63. Zhao, D. (2005). Challenges of scholarly publications on the Web to the evaluation of science--A comparison of author visibility on the Web and in print journals. Information Processing and Management, 41(6), 1403-1418. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Zhao, D., & Logan, E. (2002). Citation analysis using scientific publications on the Web as data source: A case study in the XML research area. Scientometrics, 54(3), 449-472.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Google Scholar citations and Google Web-URL citations: A multi-discipline exploratory analysis

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            Get this Article
            View Issue’s Table of Contents
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!