skip to main content
research-article

Power of Bonus in Pricing for Crowdsourcing

Authors Info & Claims
Published:15 December 2021Publication History
Skip Abstract Section

Abstract

We consider a simple form of pricing for a crowdsourcing system, where pricing policy is published a priori, and workers then decide their task acceptance. Such a pricing form is widely adopted in practice for its simplicity, e.g., Amazon Mechanical Turk, although additional sophistication to pricing rule can enhance budget efficiency. With the goal of designing efficient and simple pricing rules, we study the impact of the following two design features in pricing policies: (i) personalization tailoring policy worker-by-worker and (ii) bonus payment to qualified task completion. In the Bayesian setting, where the only prior distribution of workers' profiles is available, we first study the Price of Agnosticism (PoA) that quantifies the utility gap between personalized and common pricing policies. We show that PoA is bounded within a constant factor under some mild conditions, and the impact of bonus is essential in common pricing. These analytic results imply that complex personalized pricing can be replaced by simple common pricing once it is equipped with a proper bonus payment. To provide insights on efficient common pricing, we then study the efficient mechanisms of bonus payment for several profile distribution regimes which may exist in practice. We provide primitive experiments on Amazon Mechanical Turk, which support our analytical findings.

References

  1. Vineet Abhishek, Ian A Kash, and Peter Key. 2012. Fixed and market pricing for cloud services. In 2012 Proceedings IEEE INFOCOM Workshops. IEEE, 157--162.Google ScholarGoogle ScholarCross RefCross Ref
  2. Georgios Amanatidis, Pieter Kleer, and Guido Sch"afer. 2019. Budget-feasible mechanism design for non-monotone submodular objectives: Offline and online. In Proceedings of the 2019 ACM Conference on Economics and Computation .Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Nima Anari, Gagan Goel, and Afshin Nikzad. 2014. Mechanism design for crowdsourcing: An optimal 1--1/e competitive budget-feasible mechanism for large markets. In Proc. of FOCS .Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Eric T Anderson and Duncan I Simester. 2010. Price stickiness and customer antagonism. The quarterly journal of economics , Vol. 125, 2 (2010), 729--765.Google ScholarGoogle Scholar
  5. Moshe Babaioff, Michal Feldman, and Noam Nisan. 2006. Combinatorial agency. In Proceedings of the 7th ACM Conference on Electronic Commerce. 18--28.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Moshe Babaioff, Yannai A Gonczarowski, and Noam Nisan. 2021. The menu-size complexity of revenue approximation. Games and Economic Behavior (2021).Google ScholarGoogle Scholar
  7. Eric Balkanski and Jason D Hartline. 2016. Bayesian budget feasibility with posted pricing. In Proc. of WWW .Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Siddhartha Banerjee, Ramesh Johari, and Carlos Riquelme. 2015. Pricing in ride-sharing platforms: A queueing-theoretic approach. In Proceedings of the Sixteenth ACM Conference on Economics and Computation . 639--639.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Xiaohui Bei, Ning Chen, Nick Gravin, and Pinyan Lu. 2012. Budget feasible mechanism design: from prior-free to bayesian. In Proc. of STOC .Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Dirk Bergemann, Benjamin Brooks, and Stephen Morris. 2015. The limits of price discrimination. American Economic Review , Vol. 105, 3 (2015), 921--57.Google ScholarGoogle ScholarCross RefCross Ref
  11. Dirk Bergemann, Francisco Castro, and Gabriel Weintraub. 2019. Third-degree Price Discrimination Versus Uniform Pricing. arXiv preprint arXiv:1912.05164 (2019).Google ScholarGoogle Scholar
  12. Satyanath Bhat, Swaprava Nath, Sujit Gujar, Onno Zoeter, Yadati Narahari, and Chris Dance. 2014. A mechanism to optimally balance cost and quality of labeling tasks outsourced to strategic agents. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems. 917--924.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Michael Buhrmester, Tracy Kwang, and Samuel D Gosling. 2011. Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on psychological science (2011).Google ScholarGoogle Scholar
  14. Colin F Camerer. 2011. Behavioral game theory: Experiments in strategic interaction .Princeton university press.Google ScholarGoogle Scholar
  15. Gabriel Carroll. 2015. Robustness and linear contracts. American Economic Review (2015).Google ScholarGoogle Scholar
  16. Juan Camilo Castillo, Dan Knoepfle, and Glen Weyl. 2017. Surge pricing solves the wild goose chase. In Proceedings of the 2017 ACM Conference on Economics and Computation. 241--242.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Shuchi Chawla, Jason D Hartline, David Malec, and Balasubramanian Sivan. 2010. Sequential posted pricing and multi-parameter mechanism design. In Proc. of STOC .Google ScholarGoogle Scholar
  18. Shuchi Chawla, Yifeng Teng, and Christos Tzamos. 2020. Menu-size complexity and revenue continuity of buy-many mechanisms. In Proceedings of the 21st ACM Conference on Economics and Computation. 475--476.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Xi Chen, Ilias Diakonikolas, Anthi Orfanou, Dimitris Paparas, Xiaorui Sun, and Mihalis Yannakakis. 2015. On the complexity of optimal lottery pricing and randomized mechanisms. In 2015 IEEE 56th Annual Symposium on Foundations of Computer Science. IEEE, 1464--1479.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Yan Chen, Teck-HUa Ho, and Yong-Mi Kim. 2010. Knowledge market design: A field experiment at Google Answers. Journal of Public Economic Theory (2010).Google ScholarGoogle Scholar
  21. Yiling Chen, Yiheng Shen, and Shuran Zheng. 2020. Truthful Data Acquisition via Peer Prediction. arXiv preprint arXiv:2006.03992 (2020).Google ScholarGoogle Scholar
  22. Yanjiao Chen, Xu Wang, Baochun Li, and Qian Zhang. 2019. An incentive mechanism for crowdsourcing systems with network effects. ACM Transactions on Internet Technology (TOIT) , Vol. 19, 4 (2019), 1--21.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Vincent P Crawford. 1997. Theory and experiment in the analysis of strategic interaction. Econometric Society Monographs , Vol. 26 (1997), 206--242.Google ScholarGoogle Scholar
  24. Rachel Cummings, Nikhil R Devanur, Zhiyi Huang, and Xiangning Wang. 2020. Algorithmic price discrimination. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms. SIAM, 2432--2451.Google ScholarGoogle ScholarCross RefCross Ref
  25. George B Dantzig. 1957. Discrete-variable extremum problems. Operations research (1957).Google ScholarGoogle Scholar
  26. Alexander Philip Dawid and Allan M Skene. 1979. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied statistics (1979).Google ScholarGoogle Scholar
  27. Ludwig Dierks and Sven Seuken. 2021. Cloud pricing: the spot market strikes back. Management Science (2021).Google ScholarGoogle Scholar
  28. Lisa Drayer. 2018. Is sparkling water as hydrating as regular water? https://edition.cnn.com/2018/07/19/health/sparkling-water-hydration-drayer/index.html .Google ScholarGoogle Scholar
  29. David Easley and Arpita Ghosh. 2015. Behavioral mechanism design: Optimal crowdsourcing contracts and prospect theory. In Proceedings of the Sixteenth ACM Conference on Economics and Computation . 679--696.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Liran Einav, Chiara Farronato, Jonathan Levin, and Neel Sundaresan. 2018. Auctions versus posted prices in online markets. Journal of Political Economy , Vol. 126, 1 (2018), 178--215.Google ScholarGoogle ScholarCross RefCross Ref
  31. Adam N Elmachtoub, Vishal Gupta, and Michael Hamilton. 2018. The value of personalized pricing. Available at SSRN 3127719 (2018).Google ScholarGoogle Scholar
  32. Michal Feldman, Christos Papadimitriou, John Chuang, and Ion Stoica. 2006. Free-riding and whitewashing in peer-to-peer systems. IEEE Journal on selected areas in communications (2006).Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Arpita Ghosh and Robert Kleinberg. 2014. Optimal contest design for simple agents. In Proceedings of the fifteenth ACM conference on Economics and computation. 913--930.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Shuo Guo, Liang He, Yu Gu, Bo Jiang, and Tian He. 2014. Opportunistic flooding in low-duty-cycle wireless sensor networks with unreliable links. IEEE Trans. Comput. (2014).Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Kai Han, Yuntian He, Haisheng Tan, Shaojie Tang, He Huang, and Jun Luo. 2017. Online Pricing for Mobile Crowdsourcing with Multi-Minded Users. In Proc. of Mobihoc .Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Kai Han, He Huang, and Jun Luo. 2016. Posted pricing for robust crowdsensing. In Proc. of Mobihoc .Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Kai Han, He Huang, and Jun Luo. 2018. Quality-aware pricing for mobile crowdsensing. IEEE/ACM Transactions on Networking , Vol. 26, 4 (2018), 1728--1741.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Chien-Ju Ho, Rafael Frongillo, and Yiling Chen. 2016a. Eliciting categorical data for optimal aggregation. In Proceedings of the 30th International Conference on Neural Information Processing Systems . 2450--2458.Google ScholarGoogle Scholar
  39. Chien-Ju Ho, Aleksandrs Slivkins, Siddharth Suri, and Jennifer Wortman Vaughan. 2015. Incentivizing high quality crowdwork. In Proc. of WWW .Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Chien-Ju Ho, Aleksandrs Slivkins, and Jennifer Wortman Vaughan. 2016b. Adaptive contract design for crowdsourcing markets: Bandit algorithms for repeated principal-agent problems. Journal of Artificial Intelligence Research , Vol. 55 (2016), 317--359.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Jiali Huang, Ankur Mani, and Zizhuo Wang. 2019. The value of price discrimination in large random networks. In Proceedings of the 2019 ACM Conference on Economics and Computation. 243--244.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Grace YoungJoo Jeon, Yong-Mi Kim, and Yan Chen. 2010. Re-examining price as a predictor of answer quality in an online Q&A site. In Proceedings of the SIGCHI conference on human factors in computing systems . 325--328.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Daniel Kahneman and Amos Tversky. 2013. Prospect theory: An analysis of decision under risk. In Handbook of the fundamentals of financial decision making: Part I. World Scientific, 99--127.Google ScholarGoogle Scholar
  44. David R Karger, Sewoong Oh, and Devavrat Shah. 2014. Budget-optimal task allocation for reliable crowdsourcing systems. Operations Research , Vol. 62, 1 (2014), 1--24.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Robert E Kraut and Paul Resnick. 2012. Building successful online communities: Evidence-based social design .Mit Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Jean-Jacques Laffont and David Martimort. 2009. The theory of incentives: the principal-agent model .Princeton university press.Google ScholarGoogle Scholar
  47. Armando Levy and Tomislav Vukina. 2002. Optimal linear contracts with heterogeneous agents. European Review of Agricultural Economics , Vol. 29, 2 (2002), 205--217.Google ScholarGoogle ScholarCross RefCross Ref
  48. Leib Litman, Jonathan Robinson, and Cheskie Rosenzweig. 2015. The relationship between motivation, monetary compensation, and data quality among US-and India-based workers on Mechanical Turk. Behavior research methods (2015).Google ScholarGoogle Scholar
  49. Yang Liu and Yiling Chen. 2016. Learning to incentivize: eliciting effort via output agreement. In Proc. of AAAI .Google ScholarGoogle Scholar
  50. Qian Ma, Lin Gao, Ya-Feng Liu, and Jianwei Huang. 2018. Incentivizing Wi-Fi network crowdsourcing: A contract theoretic approach. IEEE/ACM Transactions on Networking , Vol. 26, 3 (2018), 1035--1048.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Winter Mason and Duncan J Watts. 2009. Financial incentives and the" performance of crowds". In Proceedings of the ACM SIGKDD workshop on human computation. 77--85.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Winter Mason and Duncan J Watts. 2010. Financial incentives and the performance of crowds. ACM SigKDD Explorations Newsletter (2010).Google ScholarGoogle Scholar
  53. MoralMachine. 2018. https://www.moralmachine.net/.Google ScholarGoogle Scholar
  54. Jungseul Ok, Sewoong Oh, Yunhun Jang, Jinwoo Shin, and Yung Yi. 2019. Iterative bayesian learning for crowdsourced regression. In The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 1486--1495.Google ScholarGoogle Scholar
  55. Jungseul Ok, Sewoong Oh, Jinwoo Shin, and Yung Yi. 2016. Optimality of belief propagation for crowdsourced classification. In Proc. of ICML .Google ScholarGoogle Scholar
  56. Goran Radanovic, Boi Faltings, and Radu Jurca. 2016. Incentives for effort in crowdsourcing using the peer truth serum. Proc. of TIST (2016).Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Vikas C Raykar, Shipeng Yu, Linda H Zhao, Gerardo Hermosillo Valadez, Charles Florin, Luca Bogoni, and Linda Moy. 2010. Learning from crowds. Journal of Machine Learning Research (2010).Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Shreyas Sekar. 2016. Posted pricing sans discrimination. arXiv preprint arXiv:1609.06844 (2016).Google ScholarGoogle Scholar
  59. SETI. 2018. https://www.seti.org/.Google ScholarGoogle Scholar
  60. Benson Shapiro. 2002. Is Performance-Based Pricing the Right Price for You? Verfügbar: http://hbswk. hbs. edu/item/3021. html (Zugriff am 29.03. 2007, Erstellung am 22.07. 2002) (2002).Google ScholarGoogle Scholar
  61. Yaron Singer. 2010. Budget feasible mechanisms. In Proc. of FOCS .Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Jiayi Song and Roch Guérin. 2020. Pricing (and bidding) strategies for delay differentiated cloud services. ACM Transactions on Economics and Computation (TEAC) , Vol. 8, 2 (2020), 1--58.Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Jiajun Sun and Huadong Ma. 2014. Collection-behavior based multi-parameter posted pricing mechanism for crowd sensing. In Proc. of ICC .Google ScholarGoogle ScholarCross RefCross Ref
  64. Yongxin Tong, Libin Wang, Zimu Zhou, Lei Chen, Bowen Du, and Jieping Ye. 2018. Dynamic pricing in spatial crowdsourcing: A matching-based approach. In Proceedings of the 2018 International Conference on Management of Data . 773--788.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Amazon Mechanical Turk. 2018. https://www.mturk.com/mturk/welcome.htm .Google ScholarGoogle Scholar
  66. Jan Vondrák, Chandra Chekuri, and Rico Zenklusen. 2011. Submodular function maximization via the multilinear relaxation and contention resolution schemes. In Proc. of STOC .Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Jie Zhang Zehong Hu. 2017. Optimal Posted-Price Mechanism in Microtask Crowdsourcing. In Proc. of IJCAI .Google ScholarGoogle Scholar
  68. Liang Zheng, Carlee Joe-Wong, Christopher G Brinton, Chee Wei Tan, Sangtae Ha, and Mung Chiang. 2016. On the viability of a cloud virtual service provider. ACM SIGMETRICS Performance Evaluation Review , Vol. 44, 1 (2016), 235--248.Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Liang Zheng, Carlee Joe-Wong, Chee Wei Tan, Mung Chiang, and Xinyu Wang. 2015. How to bid the cloud. ACM SIGCOMM Computer Communication Review , Vol. 45, 4 (2015), 71--84.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Power of Bonus in Pricing for Crowdsourcing

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!