Understanding Incentives in Social Computing

Social computing is an emerging field where human intelligence is harnessed in algorithmic problem solving. In particular, humans have or can efficiently gather relevant information about products, services and uncertain events and these information can be used to solve difficult problems. I have developed and analyzed such mechanisms for eliciting and aggregating dispersed information using both theoretical and experimental approaches. In this talk, I will focus on our online experiment on peer prediction mechanisms for eliciting truthful subjective feedback from participants. Peer prediction theory leverages the stochastic correlation of participants' information and designs monetary rewards to induce a truthful equilibrium among the participants. However, these mechanisms also admit uninformative equilibria where participants provide no useful information. We conduct the first comprehensive empirical evaluation of a peer prediction mechanism in a repeated setting. Our results show that, in contrast to the theory, participants are not truthful and successfully coordinate on uninformative equilibria. In the absence of peer prediction, however, most players are consistently truthful, suggesting that these mechanisms may be harmful when truthful reporting has similar cost to strategic behavior. I will also describe some of my theoretical work on analyzing the strategic behavior of participants in prediction markets, and conclude by discussing some future directions.

This talk is based on joint work with Ryan P. Adams, Yiling Chen, Rick Goldstein, Ian A. Kash, Andrew Mao, and Jie Zhang.

Xi (Alice) Gao is a PhD candidate in Computer Science at Harvard University. Her research focuses on designing and understanding algorithms and systems in social computing, crowdsourcing and human computation, using both theoretical and experimental approaches. As part of her PhD work, Alice designed and analyzed the incentives in mechanisms for eliciting and aggregating dispersed information, for applications such as eliciting subjective feedback about products and services and forecasting future events. Alice is the recipient of the Canadian NSERC Postgraduate Scholarship for Doctoral Students and the Siebel Scholarship. Before Harvard, Alice received her undergraduate degree from University of British Columbia in computer science and mathematics.