skip to main content
research-article

Efficient and Robust Emergence of Norms through Heuristic Collective Learning

Authors Info & Claims
Published:27 October 2017Publication History
Skip Abstract Section

Abstract

In multiagent systems, social norms serves as an important technique in regulating agents’ behaviors to ensure effective coordination among agents without a centralized controlling mechanism. In such a distributed environment, it is important to investigate how a desirable social norm can be synthesized in a bottom-up manner among agents through repeated local interactions and learning techniques. In this article, we propose two novel learning strategies under the collective learning framework, collective learning EV-l and collective learning EV-g, to efficiently facilitate the emergence of social norms. Extensive simulations results show that both learning strategies can support the emergence of desirable social norms more efficiently and be applicable in a wider range of multiagent interaction scenarios compared with previous work. The influence of different topologies is investigated, which shows that the performance of all strategies is robust across different network topologies. The influences of a number of key factors (neighborhood size, actions space, population size, fixed agents and isolated subpopulations) on norm emergence performance are investigated as well.

References

  1. T. Ågotnes and M. Wooldridge. 2010. Optimal social laws. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, Volume 1. 667--674.Google ScholarGoogle Scholar
  2. A. Alexander, K. Lloyd, J. Pitt, and M. Sergot. 2005. A protocol for resource sharing in norm-governed ad hoc networks. In Declarative Agent Languages and Technologies II. 221--238.Google ScholarGoogle Scholar
  3. A. Alexander, S. Marek, and P. Jeremy. 2009. Specifying norm-governed computational societies. ACM Trans. Comput. Logic 10, 1 (2009), 1.Google ScholarGoogle Scholar
  4. A. L. Barabási, R. Albert, and H. Jeong. 2000. Scale-free characteristics of random networks: The topology of the world-wide web. Physica A 281, 1 (2000), 69--77.Google ScholarGoogle ScholarCross RefCross Ref
  5. C. K. Chan, J. Y. Hao, and H. F. Leung. 2015. Reciprocal social strategy in social repeated games. In Proceedings of International Conference on Tools with Artificial Intelligence. 966--973. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. N. Criado, E. Argente, A. Garrido, J. A. Gimeno, F. Igual, V. Botti, P. Noriega, and A. Giret. 2011. Norm enforceability in electronic institutions? In Coordination, Organizations, Institutions, and Norms in Agent Systems VI. 250--267. Google ScholarGoogle ScholarCross RefCross Ref
  7. H. Franks, N. Griffiths, and A. Jhumka. 2013. Manipulating convention emergence using influencer agents. Auton. Agents Multi-Agent Syst. 26, 3 (2013), 315--353. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. K. Genter, S. Zhang, and P. Stone. 2015. Determining placements of influencing agents in a flock. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems. 247--255.Google ScholarGoogle Scholar
  9. N. Griffiths. 2008. Tags and image scoring for robust cooperation. In Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, Volume 2. 575--582.Google ScholarGoogle Scholar
  10. N. Griffiths and S. S. Anand. 2012. The impact of social placement of non-learning agents on convention emergence. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems. 1367--1368.Google ScholarGoogle Scholar
  11. N. Griffiths and M. Luck. 2010a. Changing neighbours: Improving tag-based cooperation. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems. 249--256.Google ScholarGoogle Scholar
  12. N. Griffiths and M. Luck. 2010b. Norm emergence in tag-based cooperation. In Proceedings of the 8th European Workshop on Multi-Agent Systems.Google ScholarGoogle Scholar
  13. J. Y. Hao and H. F. Leung. 2013. Achieving social optimality with influencer agents. In Complex Sciences. Springer, 140--151. Google ScholarGoogle ScholarCross RefCross Ref
  14. M. James, N. Griffiths, and M. Leeke. 2015. Convention emergence and influence in dynamic topologies. In Proceedings of International Conference on Autonomous Agents and Multiagent Systems. 1785--1786.Google ScholarGoogle Scholar
  15. S. Kapetanakis and D. Kudenko. 2002. Reinforcement learning of coordination in cooperative multiagent systems. In Proceedings of the 18th National Conference on Artificial Intelligence. 326--331.Google ScholarGoogle Scholar
  16. R. Makar, S. Mahadevan, and M. Ghavamzadeh. 2001. Hierarchical multi-agent reinforcement learning. In Proceedings of the 5th International Conference on Autonomous Agents. 246--253. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. J. Marchant, N. Griffiths, and M. Leeke. 2015. Manipulating conventions in a particle-based topology. In Proceedings of the Coordination, Organizations, Institutions and Norms in Agent Systems Workshop.Google ScholarGoogle Scholar
  18. M. Matlock and S. Sen. 2007. Effective tag mechanisms for evolving coordination. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems. ACM, 251. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. M. Matlock and S. Sen. 2009. Effective tag mechanisms for evolving cooperation. In Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems. 489--496.Google ScholarGoogle Scholar
  20. M. Mihaylov, K. Tuyls, and A. Nowé. 2014. A decentralized approach for convention emergence in multi-agent systems. Autonomous Agents and Multi-Agent Systems 28, 5 (2014), 749–778.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. J. Morales, M. Lopez-Sanchez, J. A. Rodriguez-Aguilar, M. Wooldridge, and W. Vasconcelos. 2013. Automated synthesis of normative systems. In Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems. 483--490.Google ScholarGoogle Scholar
  22. R. Polikar. 2006. Ensemble based systems in decision making. Circ. Syst. Mag. 6, 3 (2006), 21--45. Google ScholarGoogle ScholarCross RefCross Ref
  23. S. Redner. 1998. How popular is your paper an empirical study of the citation distribution. Eur. Phys. J. B 4, 2 (1998), 132--134. Google ScholarGoogle ScholarCross RefCross Ref
  24. A. Réka and A. L. Barabási. 2002. Statistical mechanics of complex networks. Rev. Mod. Phys. 74, 1 (2002), 47--97.Google ScholarGoogle ScholarCross RefCross Ref
  25. O. Sen and S. Sen. 2010. Effects of social network topology and options on norm emergence. In Coordination, Organizations, Institutions and Norms in Agent Systems V. 211--222. Google ScholarGoogle ScholarCross RefCross Ref
  26. S. Sen and S. Airiau. 2007. Emergence of norms through social learning. In Proceedings of the 16th International Joint Conference on Artificial Intelligence. 1507--1512.Google ScholarGoogle Scholar
  27. Y. Shoham and M. Tennenholtz. 1997. On the emergence of social conventions: Modeling, analysis and simulations. Artif. Intell. 94(1-2) (1997), 139--166. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. D. Villatoro, J. Sabater-Mir, and S. Sen. 2011. Social instruments for robust convention emergence. In Proceedings of the 20th International Joint Conference on Artificial Intelligence. 420--425.Google ScholarGoogle Scholar
  29. D. Villatoro, S. Sen, and J. Sabater-Mir. 2009. Topology and memory effect on convention emergence. In Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, Volume 02. 233--240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. X. F. Wang and G. R. Chen. 2003. Complex networks: Small-world, scale-free and beyond. IEEE Circ. Syst. Mag. 3, 1 (2003), 6--20. Google ScholarGoogle ScholarCross RefCross Ref
  31. T. P. Yang, Z. P. Meng, J. Y. Hao, S. Sen, and C. Yu. 2016. Accelerating norm emergence through hierarchical heuristic learning. In Proceedings of the 22th European Conference on Artificial Intelligence (ECAI’16). 1344--1352.Google ScholarGoogle Scholar
  32. H. P. Young. 1996. The economics of convention. J. Econ. Perspect. 10, 2 (1996), 105--122. Google ScholarGoogle ScholarCross RefCross Ref
  33. C. Yu, H. T. Lv, F. H. Ren, H. L. Bao, and J. Y. Hao. 2015. Hierarchical learning for emergence of social norms in networked multiagent systems. In AI 2015: Advances in Artificial Intelligence. Springer, Berlin, 630--643. Google ScholarGoogle ScholarCross RefCross Ref
  34. C. Yu, G. Z. Tan, H. T. Lv, Z. Wang, J. Meng, J. Y. Hao, and F. H. Ren. 2016. Modelling adaptive learning behaviours for consensus formation in human societies. Sci. Rep. 6 (2016). Google ScholarGoogle ScholarCross RefCross Ref
  35. C. Yu, M. J. Zhang, and F. H. Ren. 2014. Collective learning for the emergence of social norms in networked multiagent systems. IEEE Trans. Cybernet. 44, 12 (2014), 2342--2355. Google ScholarGoogle ScholarCross RefCross Ref
  36. C. Yu, M. J. Zhang, F. H. Ren, and X. D. Luo. 2013. Emergence of social norms through collective learning in networked agent societies. In Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems. 475--482.Google ScholarGoogle Scholar

Index Terms

  1. Efficient and Robust Emergence of Norms through Heuristic Collective Learning

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Autonomous and Adaptive Systems
      ACM Transactions on Autonomous and Adaptive Systems  Volume 12, Issue 4
      December 2017
      224 pages
      ISSN:1556-4665
      EISSN:1556-4703
      DOI:10.1145/3155314
      Issue’s Table of Contents

      Copyright © 2017 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 27 October 2017
      • Accepted: 1 July 2017
      • Revised: 1 January 2017
      • Received: 1 November 2015
      Published in taas Volume 12, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!