Abstract
Distributed data stream processing applications are structured as graphs of interconnected modules able to ingest high-speed data and to transform them in order to generate results of interest. Elasticity is one of the most appealing features of stream processing applications. It makes it possible to scale up/down the allocated computing resources on demand in response to fluctuations of the workload. On clouds, this represents a necessary feature to keep the operating cost at affordable levels while accommodating user-defined QoS requirements. In this article, we study this problem from a game-theoretic perspective. The control logic driving elasticity is distributed among local control agents capable of choosing the right amount of resources to use by each module. In a first step, we model the problem as a noncooperative game in which agents pursue their self-interest. We identify the Nash equilibria and we design a distributed procedure to reach the best equilibrium in the Pareto sense. As a second step, we extend the noncooperative formulation with a decentralized incentive-based mechanism in order to promote cooperation by moving the agreement point closer to the system optimum. Simulations confirm the results of our theoretical analysis and the quality of our strategies.
- Joao Gama and Mohamed Medhat Gaber. 2007. Learning from Data Streams: Processing Techniques in Sensor Networks (1 ed.). Google Scholar
Digital Library
- 2015. FastFlow (FF). Retrieved from http://http://calvados.di.unipi.it/fastflow/.Google Scholar
- M. Akdere, C. Ç. Bilgin, O. Gerdaneri, I. Korpeoglu, Ö. Ulusoy, and U. Çetintemel. 2006. A comparison of epidemic algorithms in wireless sensor networks. Comput. Commun. 29, 13--14 (Aug. 2006), 2450--2457. Google Scholar
Digital Library
- T. Alpcan and L. Pavel. 2009. Nash equilibrium design and optimization. In Proceedings of the International Conference on Game Theory for Networks, 2009 (GameNets’09). 164--170. Google Scholar
Digital Library
- H. Andrade, B. Gedik, and D. Turaga. 2014. Fundamentals of Stream Processing. Cambridge University Press. Cambridge Books Online. Google Scholar
Digital Library
- H. Andrade, B. Gedik, K. L. Wu, and P. S. Yu. 2011. Processing high data rate streams in system S. J. Parallel Distrib. Comput. 71, 2 (Feb. 2011), 145--156. Google Scholar
Digital Library
- D. Ardagna, B. Panicucci, and M. Passacantando. 2013. Generalized Nash equilibria for the service provisioning problem in cloud systems. IEEE Trans. Services Comput. 6, 4 (Oct. 2013), 429--442. Google Scholar
Digital Library
- S. Babu and J. Widom. 2001. Continuous queries over data streams. SIGMOD Rec. 30, 3 (Sept. 2001), 109--120. Google Scholar
Digital Library
- C. Bertolli, G. Mencagli, and M. Vanneschi. 2009. Adaptivity in risk and emergency management applications on pervasive grids. In Proceedings of the 2009 10th International Symposium on Pervasive Systems, Algorithms, and Networks (ISPAN). 550--555. DOI:http://dx.doi.org/10.1109/I-SPAN.2009.92 Google Scholar
Digital Library
- C. Bertolli, G. Mencagli, and M. Vanneschi. 2010. A cost model for autonomic reconfigurations in high-performance pervasive applications. In Proceedings of the 4th ACM International Workshop on Context-Awareness for Self-Managing Systems (CASEMANS’10). ACM, New York, NY, Article 3, 10 pages. DOI:http://dx.doi.org/10.1145/1858367.1858370 Google Scholar
Digital Library
- A. Buades, B. Coll, and J.-M. Morel. 2005. A non-local algorithm for image denoising. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) - Volume 2. IEEE Computer Society, Washington, DC, 60--65. Google Scholar
Digital Library
- R. C. Fernandez, M. Migliavacca, E. Kalyvianaki, and P. Pietzuch. 2013. Integrating scale out and fault tolerance in stream processing using operator state management. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (SIGMOD’13). ACM, New York, NY, 725--736. Google Scholar
Digital Library
- S. Chaisiri, B.-S. Lee, and D. Niyato. 2012. Optimization of resource provisioning cost in cloud computing. IEEE Trans. Serv. Comput. 5, 2 (April 2012), 164--177. Google Scholar
Digital Library
- G. Chalkiadakis, E. Elkind, and M. Wooldridge. 2012. Cooperative game theory: Basic concepts and computational challenges. IEEE Intelligent Syst. 27, 3 (2012), 86--90. Google Scholar
Digital Library
- G. Cugola and A. Margara. 2012. Processing flows of information: From data stream to complex event processing. ACM Comput. Surv. 44, 3, Article 15 (June 2012), 62 pages. Google Scholar
Digital Library
- P. Dubey. 1986. Inefficiency of Nash equilibria. Math. Oper. Res. 11, 1 (1986), 1--8. Google Scholar
Digital Library
- B. Espinasse, G. Picolet, and E. Chouraqui. 1997. Negotiation support systems: A multi-criteria and multi-agent approach. Eur. J. Oper. Res. 103, 2 (1997), 389--409.Google Scholar
Cross Ref
- R. Gayathri and R. S. Sabeenian. 2013. A performance analysis of efficient schemes and algorithms in image denoising procedures. In Proceedings of the 2013 International Conference on Computer Communication and Informatics (ICCCI’13). 1--5.Google Scholar
- B. Gedik, S. Schneider, M. Hirzel, and K.-L. Wu. 2014. Elastic scaling for data stream processing. IEEE Parallel Distr. Syst. 25, 6 (June 2014), 1447--1463. Google Scholar
Digital Library
- J. Geibig and D. Bradler. 2010. Self-organized aggregation in irregular wireless networks. In Proceedings of the 2010 IFIP Wireless Days (WD’10), 1--7.Google Scholar
- A. Gohad, N. C. Narendra, and P. Ramachandran. 2013. Cloud pricing models: A survey and position paper. In 2013 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM’13). 1--8.Google Scholar
- E. R. Gomes, Q. B. Vo, and R. Kowalczyk. 2012. Pure exchange markets for resource sharing in federated clouds. Concurrency Comput. Pract. Exper. 24, 9 (2012), 977--991. Google Scholar
Digital Library
- H. González-Vélez and M. Leyton. 2010. A survey of algorithmic skeleton frameworks: High-level structured parallel programming enablers. Softw. Pract. Exper. 40, 12 (Nov. 2010), 1135--1160. Google Scholar
Digital Library
- V. Gulisano, R. Jimenez-Peris, M. Patino-Martinez, C. Soriente, and P. Valduriez. 2012. StreamCloud: An elastic and scalable data streaming system. IEEE Trans. Parallel Distrib. Syst. 23, 12 (Dec. 2012), 2351--2365. Google Scholar
Digital Library
- T. Heinze, Z. Jerzak, G. Hackenbroich, and C. Fetzer. 2014. Latency-aware elastic scaling for distributed data stream processing systems. In Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems (DEBS’14). ACM, New York, NY, 13--22. Google Scholar
Digital Library
- W. Hummer, B. Satzger, and S. Dustdar. 2013. Elastic stream processing in the cloud. Wiley Interdisc. Rev. Data Mining Knowl. Discov. 3, 5 (2013), 333--345.Google Scholar
Digital Library
- Amazon Inc. 2008. Amazon Elastic Compute Cloud (Amazon EC2). Amazon Inc. Retrieved from http://aws.amazon.com/ec2/#pricing. http://aws.amazon.com/ec2/#pricing.Google Scholar
- D. Kempe, A. Dobra, and J. Gehrke. 2003. Gossip-based computation of aggregate information. In Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science (FOCS’03). IEEE Computer Society, 482. Google Scholar
Digital Library
- Y.-K. Kwok, K. Hwang, and S. Song. 2007. Selfish grids: Game-theoretic modeling and NAS/PSA benchmark evaluation. IEEE Trans. Parallel Distrib. Syst. 18, 5 (May 2007), 621--636. Google Scholar
Digital Library
- A. M. Law and D. M. Kelton. 1999. Simulation Modeling and Analysis (3rd ed.). McGraw-Hill Higher Education. Google Scholar
Digital Library
- C. Leopold. 2001. Parallel and Distributed Computing: A Survey of Models, Paradigms and Approaches. John Wiley & Sons, New York, NY. Google Scholar
Digital Library
- H. Li, C. Wu, Z. Li, and F. C. M. Lau. 2013. Profit-maximizing virtual machine trading in a federation of selfish clouds. In Proceedings of the 2013 IEEE INFOCOM, 25--29.Google Scholar
Cross Ref
- M. Lin, A. Wierman, L. L. H. Andrew, and E. Thereska. 2013. Dynamic right-sizing for power-proportional data centers. IEEE/ACM Trans. Netw. 21, 5 (Oct. 2013), 1378--1391. Google Scholar
Digital Library
- B. Lohrmann, P. Janacik, and O. Kao. 2015. Elastic stream processing with latency guarantees. In Proceedings of the 2015 IEEE 35th International Conference on Distributed Computing Systems (ICDCS). Columbus, OH, 399--410. DOI:10.1109/ICDCS.2015.48Google Scholar
Cross Ref
- M. Maggio, H. Hoffmann, A. V. Papadopoulos, J. Panerati, M. D. Santambrogio, A. Agarwal, and A. Leva. 2012. Comparison of decision-making strategies for self-optimization in autonomic computing systems. ACM Trans. Auton. Adapt. Syst. 7, 4, Article 36 (Dec. 2012), 32 pages. Google Scholar
Digital Library
- R. Makhloufi, G. Bonnet, G. Doyen, and D. Gaïti. 2009. Decentralized aggregation protocols in peer-to-peer networks: A survey. In Modelling Autonomic Communications Environments, J. C. Strassner and Y. M. Ghamri-Doudane (Eds.). Lecture Notes in Computer Science, Vol. 5844. Springer, Berlin, 111--116. Google Scholar
Digital Library
- R. Makhloufi, G. Doyen, G. Bonnet, and D. Gaïti. 2014. A survey and performance evaluation of decentralized aggregation schemes for autonomic management. Int. J. Netw. Manage. 24, 6 (2014), 469--498. Google Scholar
Digital Library
- D. Meilander, S. Kottinger, and S. Gorlatch. 2013. A scalability model for distributed resource management in real-time online applications. In Proceedings of the 2013 42nd International Conference on Parallel Processing (ICPP’13). 763--772. DOI:http://dx.doi.org/10.1109/ICPP.2013.90 Google Scholar
Digital Library
- G. Mencagli. 2012. A Control-Theoretic Methodology for Controlling Adaptive Structured Parallel Computations. Ph.D Thesis, Department of Computer Science, University of Pisa, Italy.Google Scholar
- G. Mencagli and M. Vanneschi. 2011. QoS-control of structured parallel computations: A predictive control approach. In Proceedings of the 2011 IEEE 3rd International Conference on Cloud Computing Technology and Science (CloudCom’11). 296--303. Google Scholar
Digital Library
- G. Mencagli, M. Vanneschi, and E. Vespa. 2013a. Control-theoretic adaptation strategies for autonomic reconfigurable parallel applications on cloud environments. In 2013 International Conference on High Performance Computing and Simulation (HPCS’13). 11--18.Google Scholar
- G. Mencagli, M. Vanneschi, and E. Vespa. 2013b. Reconfiguration stability of adaptive distributed parallel applications through a cooperative predictive control approach. In Proceedings of the 19th International Conference on Parallel Processing (Euro-Par’13). Springer-Verlag, Berlin, 329--340. Google Scholar
Digital Library
- G. Mencagli, M. Vanneschi, and E. Vespa. 2014. A cooperative predictive control approach to improve the reconfiguration stability of adaptive distributed parallel applications. ACM Trans. Auton. Adapt. Syst. 9, 1, Article 2 (March 2014), 27 pages. Google Scholar
Digital Library
- J. Nash. 1951. Non-cooperative games. Ann. Math. 54, 2 (Sept. 1951), 286--295.Google Scholar
Cross Ref
- D. Niyato, K. Zhu, and P. Wang. 2011. Cooperative virtual machine management for multi-organization cloud computing environment. In Proceedings of the 5th International ICST Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS’11). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), ICST, Brussels, Belgium, 528--537. Google Scholar
Digital Library
- A. Núñez, J. L. Vázquez-Poletti, A. C. Caminero, G. G. Castañé, J. Carretero, and I. M. Llorente. 2012. iCanCloud: A flexible and scalable cloud infrastructure simulator. J. Grid Comput. 10, 1 (March 2012), 185--209. Google Scholar
Digital Library
- J. Park and M. van der Schaar. 2010. A game theoretic analysis of incentives in content production and sharing over peer-to-peer networks. IEEE J. Selected Topics Signal Process. 4, 4 (Aug. 2010), 704--717.Google Scholar
- A. G. Prieto and R. Stadler. 2007. A-GAP: An adaptive protocol for continuous network monitoring with accuracy objectives. IEEE Trans. Netw. Serv. Manag. 4, 1 (June 2007), 2--12. Google Scholar
Digital Library
- T. Reuter and P. Cimiano. 2012. Event-based classification of social media streams. In Proceedings of the 2nd ACM International Conference on Multimedia Retrieval (ICMR’12). ACM, New York, NY, Article 22, 8 pages. Google Scholar
Digital Library
- W. Rogerson. 1994. A theory of incentives in procurement and regulation by Jean-Jacques Laffont; Jean Tirole. J. Political Econ. 102, 2 (1994), 397--402.Google Scholar
Cross Ref
- C. U. Saraydar, N. B. Mandayam, and D. Goodman. 2002. Efficient power control via pricing in wireless data networks. IEEE Trans. Commun. 50, 2 (Feb. 2002), 291--303.Google Scholar
Cross Ref
- R. Scattolini. 2009. Architectures for distributed and hierarchical model predictive control - a review. J. Process Control 19, 5 (2009), 723--731.Google Scholar
Cross Ref
- Y. Tang and B. Gedik. 2013. Autopipelining for data stream processing. IEEE Trans. Parallel Distrib. Syst. 24, 12 (Dec. 2013), 2344--2354. Google Scholar
Digital Library
- N. Vorobjov. 1994. Foundations of Game Theory - Noncooperative Games. Birkhäuser. I--VI, 1--496 pages.Google Scholar
- Y. Wu and K.-L. Tan. 2015. ChronoStream: Elastic stateful stream computation in the cloud. In Proceedings of the 2015 IEEE 31st International Conference on Data Engineering (ICDE’15). 723--734.Google Scholar
Cross Ref
- L. Yang, J. Cao, Y. Yuan, T. Li, A. Han, and A. Chan. 2013. A framework for partitioning and execution of data stream applications in mobile cloud computing. SIGMETRICS Perform. Eval. Rev. 40, 4 (April 2013), 23--32. Google Scholar
Digital Library
- D. Ye and J. Chen. 2013. Non-cooperative games on multidimensional resource allocation. Future Gener. Comput. Syst. 29, 6 (Aug. 2013), 1345--1352. Google Scholar
Digital Library
- Q. Yuan, Z. Liu, J. Peng, X. Wu, J. Li, F. Han, Q. Li, W. Zhang, X. Fan, and S. Kong. 2011. A leasing instances based billing model for cloud computing. In Proceedings of the 6th International Conference on Advances in Grid and Pervasive Computing (GPC’11). Springer-Verlag, Berlin, 33--41. Google Scholar
Digital Library
- T. Zhang and P. Xiao. 2014. A novel resource pricing mechanism based on multi-player gaming model in cloud environments. J. Softw. 9, 6 (2014), 1574--1580.Google Scholar
Cross Ref
Index Terms
A Game-Theoretic Approach for Elastic Distributed Data Stream Processing
Recommendations
Elastic Stream Computing with Clouds
CLOUD '11: Proceedings of the 2011 IEEE 4th International Conference on Cloud ComputingStream computing, also known as data stream processing, has emerged as a new processing paradigm that processes incoming data streams from tremendous numbers of sensors in a real-time fashion. Data stream applications must have low latency even when the ...
A game-theoretic approach to the financial benefits of infrastructure-as-a-service
Financial benefits are an important factor when cloud infrastructure is considered to meet processing demand. The dynamics of on-demand pricing and service usage are investigated in a two-stage game model for a monopoly Infrastructure-as-a-Service (IaaS)...
Pricing strategy of ecological industry chain based on game theory
The pricing decision of three stages ecological industry chain of which is consisted of manufacturer producing mainproduct and byproduct, mainproduct seller and byproduct buyer is studied. Four kinds of pricing decision were discussed: manufacturer ...






Comments