skip to main content
article

A stochastic model for the evolution of the Web allowing link deletion

Published:01 May 2006Publication History
Skip Abstract Section

Abstract

Recently several authors have proposed stochastic evolutionary models for the growth of the Web graph and other networks that give rise to power-law distributions. These models are based on the notion of preferential attachment, leading to the “rich get richer” phenomenon. We present a generalization of the basic model by allowing deletion of individual links and show that it also gives rise to a power-law distribution. We derive the mean-field equations for this stochastic model and show that, by examining a snapshot of the distribution at the steady state of the model, we are able to determine the extent to which link deletion has taken place and estimate the probability of deleting a link. Applying our model to actual Web graph data provides evidence of the extent to which link deletion has occurred. We also discuss a problem that frequently arises in estimating the power-law exponent from empirical data and a few possible methods for dealing with this, indicating our preferred approach. Using this approach our analysis of the data suggests a power-law exponent of approximately 2.15 for the distribution of inlinks in the Web graph, rather than the widely published value of 2.1.

References

  1. Adamic, L. and Huberman, B. 2001. The Web's hidden order. Commun. ACM 44, 9, 55--59. Google ScholarGoogle Scholar
  2. Albert, R. and Barabási, A.-L. 2002. Statistical mechanics of complex networks. Rev. Mod. Phys. 74, 47--97. Google ScholarGoogle Scholar
  3. Barabási, A.-L. and Albert, R. 1999. Emergence of scaling in random networks. Science 286, 509--512.Google ScholarGoogle Scholar
  4. Bornholdt, S. and Ebel, H. 2001. World Wide Web scaling exponent from Simon's 1955 model. Phys. Rev. E 64, 035104-1--035104-4.Google ScholarGoogle Scholar
  5. Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, A., Stata, R., Tomkins, A., and Wiener, J. 2000. Graph structure in the Web. Comput. Netw. 33, 309--320. Google ScholarGoogle Scholar
  6. Cooper, C. and Frieze, A. 2003. A general model of Web graphs. Rand. Struct. Algorith. 22, 311--335. Google ScholarGoogle Scholar
  7. Dill, S., Kumar, R., McCurley, K., Rajagopalan, S., Sivakumar, D., and Tomkins, A. 2002. Self-similarity in the Web. ACM Trans. Internet Tech. 2, 205--223. Google ScholarGoogle Scholar
  8. Dorogovtsev, S. and Mendes, J. 2001. Scaling properties of scale-free evolving networks: Continuous approach. Phys. Rev. E 35, 056125.Google ScholarGoogle Scholar
  9. Dorogovtsev, S. and Mendes, J. 2002. Evolution of networks. Adv. Phys. 51, 1079--1187.Google ScholarGoogle Scholar
  10. Drees, H., de Haan, L., and Resnick, S. 2000. How to make a Hill plot. Ann. Stat. 28, 254--274.Google ScholarGoogle Scholar
  11. Fenner, T., Levene, M., and Loizou, G. 2005. A stochastic evolutionary model exhibiting power-law behaviour with an exponential cutoff. Physica A 335, 641--656.Google ScholarGoogle Scholar
  12. Krapivsky, P. and Redner, S. 2002. A statistical physics perspective on Web growth. Comput. Netw. 39, 261--276.Google ScholarGoogle Scholar
  13. Levene, M., Fenner, T., Loizou, G., and Wheeldon, R. 2002. A stochastic model for the evolution of the Web. Comput. Netw. 39, 277--287.Google ScholarGoogle Scholar
  14. Newman, M. 2003. The structure and function of complex networks. SIAM Rev. 45, 167--256.Google ScholarGoogle Scholar
  15. Nicholls, P. 1989. Bibliometric modeling processes and the empirical validity of Lotka's law. J. Amer. Soc. Inform. Sci. 40, 379--385.Google ScholarGoogle Scholar
  16. Notess, G. 2002. The wayback machine: The Web's archive. Online 26, 59--61.Google ScholarGoogle Scholar
  17. Pennock, D., Flake, G., Lawrence, S., Glover, E., and Giles, C. 2002. Winners don't take all: Characterizing the competition for links on the web. Proc. Nat. Acad. Sci. U.S.A. 99, 5207--5211.Google ScholarGoogle Scholar
  18. Ross, S. 1983. Introduction to Stochastic Dynamic Programming. Academic Press, New York, NY. Google ScholarGoogle Scholar
  19. Schroeder, M. 1991. Fractals, Chaos, Power Laws: Minutes from an Infinite Paradise. W. H. Freeman, New York, NY.Google ScholarGoogle Scholar
  20. Simon, H. 1955. On a class of skew distribution functions. Biometrika 42, 425--440.Google ScholarGoogle Scholar

Index Terms

  1. A stochastic model for the evolution of the Web allowing link deletion

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!