Abstract
The use of resources in multiagent learning systems is a relevant research problem, with a number of applications in resource allocation, communication and synchronization. Multiagent distributed resource allocation requires that agents act on limited, localized information with minimum communication overhead in order to optimize the distribution of available resources. When requirements and constraints are dynamic, learning agents may be needed to allow for adaptation. One way of accomplishing learning is to observe past outcomes, using such information to improve future decisions. When limits in agents' memory or observation capabilities are assumed, one must decide on how large should the observation window be. We investigate how this decision influences both agents' and system's performance in the context of a special class of distributed resource allocation problems, namely dispersion games. We show by using several numerical experiments over a specific dispersion game (the Minority Game) that in such scenario an agent's performance is non-monotonically correlated with her memory size when all other agents are kept unchanged. We then provide an information-theoretic explanation for the observed behaviors, showing that a downward causation effect takes place.
- Andrecut, M. and Ali, M. 2001. Q learning in the minority game. Phys. Rev. E 64, 67--103.Google Scholar
Cross Ref
- Araújo, R. M. and Lamb, L. C. 2004. Towards understanding the role of learning models in the dynamics of the minority game. In Proceedings of 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'04). 727--731. Google Scholar
Digital Library
- Araújo, R. M. and Lamb, L. C. 2007. A information-theoretic analysis of memory bounds in a distributed resource allocation mechanism. In Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI'07). AAAI Press, 212--217. Google Scholar
Digital Library
- Arthur, W. B. 1994. Inductive reasoning and bounded rationality. Am. Econ. Rev. 84, 406--411.Google Scholar
- Bazzan, A., Bordini, R., Vicari, R. M., and Wahle, J. 2000. Wayward agents in a commuting scenario (personalities in the minority game). In Proceedings of the 4th International Conference on Multi-Agent Systems. IEEE, 55--62. Google Scholar
Digital Library
- Bonabeau, E. 1999. Editor's introduction: Stigmergy. Artif. Life 5, 2, 95--96. Google Scholar
Digital Library
- Bonabeau, E., Dorigo, M., and Theraulaz, G. 1999. Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press. Google Scholar
Digital Library
- Carbonell, J. 1990. Machine Learning: Paradigms and Methods. Cambridge: MIT Press. Google Scholar
Digital Library
- Cavagna, A. 1999. Irrelevance of memory in the minority game. Phys. Rev. E 59, 3783--3786.Google Scholar
Cross Ref
- Challet, D., Chessa, A., Marsili, M., and Zhang, Y.-C. 2001. From minority games to real markets. Quantit. Finan. 1, 168--176.Google Scholar
Cross Ref
- Challet, D., Marsili, M., and Zhang, Y.-C. 2004. Minority Games. Oxford University Press.Google Scholar
- Challet, D. and Zhang, Y.-C. 1997. Emergence of cooperation and organization in an evolutionary game. Physica A 246. 407--418.Google Scholar
- Challet, D. and Zhang, Y.-C. 1998. On the minority game: Analytical and numerical studies. Physica A 256, 514--532.Google Scholar
Cross Ref
- Chen, Y. and Gazzale, R. S. 2004. When does learning in games generate convergence to Nash equilibria? the role of supermodularity in an experimental setting. Am. Econo. Rev. 94, 5, 1505--1535.Google Scholar
Cross Ref
- Chow, F. and Chau, H. 2003. Multiple choice minority game. Physica A 319, 601--615.Google Scholar
Cross Ref
- Crandall, J. W. and Goodrich, M. A. 2005. Learning to compete, compromise, and cooperate in repeated general-sum games. In Proceedings of the 22nd International Conference on Machine Learning (ICML'05). ACM Press, 161--168. Google Scholar
Digital Library
- de Cara, A. R., Pla, O., and Guinea, F. 1999. Competition, efficiency and collective behavior in the “El Farol” bar model. Eur. Phys. J. B 10, 187--191.Google Scholar
Cross Ref
- Dietterich, T. G. 2003. Machine learning. In Nature Encyclopedia of Cognitive Science. Macmillan, London.Google Scholar
- Edmonds, B. and Moss, S. 1997. Modelling bounded rationality using evolutionary techniques. Proceedings of the AISB'97 Workshop on Evolutionary Computation, 31--42. Google Scholar
Digital Library
- Farmer, J. D. 1999. Physicists attempt to scale the ivory towers of finance. Computing in Science and Engineering 1, 6, 26--39. Google Scholar
Digital Library
- Fogel, D. 2000. Evolutionary Computation: Toward a New Philosophy of Machine Intelligence, 2nd ed. IEEE Press, New York. Google Scholar
Digital Library
- Geisendorf, S. 2000. Modeling bounded rationality by genetic algorithms. In Proceedings of the 24th International Conference of Agricultural Economists (IAAE). 31--42.Google Scholar
- Gelenbe, E., Xu, Z., and Seref, E. 1999. Cognitive packet networks. In Proceedings of the 11th IEEE International Conference on Tools with Artificial Intelligence. Google Scholar
Digital Library
- Grenager, T., Powers, R., and Shoham, Y. 2002. Dispersion games: general definitions and some specific learning results. In Proceedings of the 18th National Conference on Artificial Intelligence (AAAI'02). AAAI Press, 398--403. Google Scholar
Digital Library
- Hu, J. and Wellman, M. P. 1998. Online learning about other agents in a dynamic multiagent system. In Proceedings of Agents-98. ACM, 239--246. Google Scholar
Digital Library
- Kahneman, D. 2003. Maps of bounded rationality: Psychology for behavioral economics. The Am. Econo. Rev. 93, 5, 1449--1475.Google Scholar
Cross Ref
- Li, Y., Riolo, R., and Savit, R. 2000. Evolution in minority games II: Games with variable strategy spaces. Physica A 276. 265--283.Google Scholar
- Li, Y., VanDeemen, A., and Savit, R. 2000. The minority game with variable payoffs. Physica A 284, 501--503.Google Scholar
Cross Ref
- MacKay, D. J. 2003. Information Theory, Inference & Learning Algorithms. Cambridge University Press. Google Scholar
Digital Library
- Manuca, R., Li, Y., Riolo, R., and Savit, R. 2000. The structure of adaptive competition in minority games. Physica A 282. 559--608.Google Scholar
- Michalski, R. S., Carbonell, J. G., and Mitchell, T. M. 1983. Machine Learning: An Artificial Intelligence Approach. Morgan Kaufmann, Palo Alto. Google Scholar
Digital Library
- Mitchell, T. 1997. Machine Learning. McGraw-Hill. Google Scholar
Digital Library
- Moelbert, S. and Rios, P. D. L. 2002. The local minority game. Physica A 303, 217--225.Google Scholar
Cross Ref
- Moro, E. 2004. The minority game: an introductory guide. In Advances in Condensed Matter and Statistical Physics, E. Korutcheva and R. Cuerno, Eds. Nova Science Publishers.Google Scholar
- Powers, R. and Shoham, Y. 2005. Learning against opponents with bounded memory. In Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI'05). 817--822. Google Scholar
Digital Library
- Rubinstein, A. 1998. Modeling Bounded Rationality. Zeuthen Lecture Book Series. The MIT Press, Cambridge, Massachussets.Google Scholar
- Savit, R., Manuca, R., and Riolo, R. 1999. Adaptive competition, market efficiency, phase transitions and spin-glasses. Physical Review Letters 82, 2203--2206.Google Scholar
Cross Ref
- Shoham, Y., Powers, R., and Grenager, T. 2004. On the agenda(s) of research on multi-agent learning. In Proceedings of AAAI Fall Symposium on Artificial Multi-Agent Learning. Technical Report FS-04-02.Google Scholar
- Simon, H. 1955. A behavioral model of rational choice. The Quart. J. Econ. 69.Google Scholar
- Sysi-Aho, M., Chakraborti, A., and Kasti, K. 2003. Intelligent minority game with genetic-crossover strategies. Eur. Phys. J. B 34, 00234, 373--377.Google Scholar
Cross Ref
- Theraulaz, G. and Bonabeau, E. 1999. A brief history of stigmergy. Artif. Life 5, 2, 97--116. Google Scholar
Digital Library
- Vidal, J. M. and Durfee, E. H. 1997. Agents learning about agents: A framework and analysis. In Proceedings of the 14th National Conference on Artificial Intelligence (AAAI'97), Multiagent Learning Workshop.Google Scholar
- Wang, T. and Liu, J. 2005a. Evaluating the Minority Game strategy in agent role assignments. In Proceedings of the 4th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS'05). 1347-1348. Google Scholar
Digital Library
- Wang, T. and Liu, J. 2005b. The Minority Game Strategy in Team Competition: How and When? In Proceedings of the 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology. 587--594. Google Scholar
Digital Library
Index Terms
On the use of memory and resources in minority games
Recommendations
Learning in groups of traffic signals
Computer science in general, and artificial intelligence and multiagent systems in particular, are part of an effort to build intelligent transportation systems. An efficient use of the existing infrastructure relates closely to multiagent systems as ...
Opportunities for multiagent systems and multiagent reinforcement learning in traffic control
The increasing demand for mobility in our society poses various challenges to traffic engineering, computer science in general, and artificial intelligence and multiagent systems in particular. As it is often the case, it is not possible to provide ...
Learning about other agents in a dynamic multiagent system
We analyze the problem of learning about other agents in a class of dynamic multiagent systems, where performance of the primary agent depends on behavior of the others. We consider an online version of the problem, where agents must learn models of the ...






Comments