When (ish) is My Bus?: User-centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems

ABSTRACT

Users often rely on realtime predictions in everyday contexts like riding the bus, but may not grasp that such predictions are subject to uncertainty. Existing uncertainty visualizations may not align with user needs or how they naturally reason about probability. We present a novel mobile interface design and visualization of uncertainty for transit predictions on mobile phones based on discrete outcomes. To develop it, we identified domain specific design requirements for visualizing uncertainty in transit prediction through: 1) a literature review, 2) a large survey of users of a popular realtime transit application, and 3) an iterative design process. We present several candidate visualizations of uncertainty for realtime transit predictions in a mobile context, and we propose a novel discrete representation of continuous outcomes designed for small screens, quantile dotplots. In a controlled experiment we find that quantile dotplots reduce the variance of probabilistic estimates by ~1.15 times compared to density plots and facilitate more confident estimation by end-users in the context of realtime transit prediction scenarios.

References

1. Nicholas J Barrowman and Ransom A Myers. 2003. Raindrop Plots: A New Way to Display Collections of Likelihoods and Distributions. The American Statistician 57, 4: 268-274. http://doi.org/10.1198/0003130032369
2. Sarah Belia, Fiona Fidler, Jennifer Williams, and Geoff Cumming. 2005. Researchers misunderstand confidence intervals and standard error bars. Psychological methods 10, 4: 389-96. http://doi.org/10.1037/1082--989X.10.4.389
3. Jacques Bertin. 1983. Semiology of Graphics: Diagrams, Networks, Maps.
4. H Choo and S L Franconeri. 2014. Enumeration of small collections violates Weber's law. Psychonomic bulletin & review 21, 1: 93-9. http://doi.org/10.3758/s13423-013-0474--4
5. Kristin A Cook and James J Thomas. 2005. Illuminating the path: The research and development agenda for visual analytics. Pacific Northwest National Laboratory (PNNL), Richland, WA. Retrieved from http://vis.pnnl.gov/pdf/RD_Agenda_VisualAnalytics.p df
6. Michael Correll and Michael Gleicher. 2014. Error bars considered harmful: Exploring alternate encodings for mean and error. IEEE transactions on visualization and computer graphics 20, 12: 2142-2151. http://doi.org/10.1109/TVCG.2014.2346298
7. Geoff Cumming. 2009. Inference by eye: reading the overlap of independent confidence intervals. Statistics in medicine 28, 2: 205-220.
8. Eric D. Feigelson and G. Jogesh Babu (eds.). 1992. Statistical Challenges in Modern Astronomy. Springer NY, NY, NY. Retrieved September 25, 2015 from http://www.springerlink.com/index/10.1007/978--14613--9290--3
9. Brian Ferris, Kari Watkins, and Alan Borning. 2010. OneBusAway: results from providing real-time arrival information for public transit. Proceedings of the 28th international conference on Human factors in computing systems CHI '10, ACM Press, 1807. Retrieved July 13, 2015 from http://dl.acm.org/citation.cfm?id=1753326.1753597
10. Richard Finger and Ann M. Bisantz. 2002. Utilizing graphical formats to convey uncertainty in a decisionmaking task. Theoretical Issues in Ergonomics Science 3, 1: 1-25. http://doi.org/10.1080/14639220110110324
11. R. Garcia-Retamero and E. T. Cokely. 2013. Communicating Health Risks With Visual Aids. Current Directions in Psychological Science 22, 5: 392-399. http://doi.org/10.1177/0963721413491570
12. Gerd Gigerenzer and Ulrich Hoffrage. 1995. How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review 102, 4: 684- 704. http://doi.org/10.1037//0033--295X.102.4.684
13. Ernst von Glasersfeld. 1982. Subitizing: The role of figural patterns in the development of numerical concepts. Archives de Psychologie 50, 194: 191-218.
14. Theresia Gschwandtnei, Markus Bogl, Paolo Federico, and Silvia Miksch. 2016. Visual Encodings of Temporal Uncertainty: A Comparative User Study. IEEE Transactions on Visualization and Computer Graphics 22, 1: 539-548. Retrieved January 8, 2016 from http://www.ncbi.nlm.nih.gov/pubmed/26529717
15. Jeffrey Heer, Nicholas Kong, and Maneesh Agrawala. 2009. Sizing the horizon: the effects of chart size and layering on the graphical perception of time series visualizations. Proceedings of the 27th international conference on Human factors in computing systems CHI 09, ACM Press, 1303. http://doi.org/10.1145/1518701.1518897
16. U Hoffrage and G Gigerenzer. 1998. Using natural frequencies to improve diagnostic inferences. Academic medicine?: journal of the Association of American Medical Colleges 73, 5: 538-540. http://doi.org/10.1097/00001888--199805000-00024
17. Jessica Hullman, Paul Resnick, and Eytan Adar. 2015. Hypothetical Outcome Plots Outperform Error Bars and Violin Plots for Inferences about Reliability of Variable Ordering. PloS one 10, 11. http://doi.org/10.1371/journal.pone.0142444
18. Harald Ibrekk and M.Granger Morgan. 1987. Graphical Communication of Uncertain Quantities to Nontechnical People. Risk Analysis 7, 4: 519-529. http://doi.org/10.1111/j.1539--6924.1987.tb00488.x
19. S. Joslyn and J. LeClerc. 2013. Decisions With Uncertainty: The Glass Half Full. Current Directions in Psychological Science 22, 4: 308-315. http://doi.org/10.1177/0963721413481473
20. Harshath JR. 2015. Redesigning the OneBusAway Mobile Experience.
21. Malte F Jung, David Sirkin, and Martin Steinert. 2015. Displayed Uncertainty Improves Driving Experience and Behavior?: The Case of Range Anxiety in an Electric Car. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15): 2201-2210. http://doi.org/10.1145/2702123.2702479
22. Peter Kampstra. 2008. Beanplot: A Boxplot Alternative for Visual
23. nComparison of Distributions. Journal of Statistical Software 28, code snippet 1: 1-9. Retrieved from http://www.jstatsoft.org/v28/c01/paper
24. 23. Matthew Kay, Dan Morris, Mc Schraefel, and Julie A Kientz. 2013. There's No Such Thing as Gaining a Pound: Reconsidering the Bathroom Scale User Interface. Ubicomp '13: 401-410.
25. 24. John K. Kruschke. 2010. Bayesian data analysis. Wiley Interdisciplinary Reviews: Cognitive Science 1, 5: 658- 676. http://doi.org/10.1002/wcs.72
26. 25. John K. Kruschke. 2011. Doing Bayesian Data Analysis. Elsevier Inc.
27. 26. Jock Mackinlay. 1986. Automating the design of graphical presentations of relational information. ACM Transactions on Graphics 5, 2: 110-141. Retrieved May 26, 2015 from http://dl.acm.org/citation.cfm?id=22949.22950
28. 27. S. S. Pak, J. B. Hutchinson, and N. B. Turk-Browne. 2014. Intuitive statistics from graphical representations of data. Journal of Vision 14, 10: 1361-1361. Retrieved January 8, 2016 from http://jov.arvojournals.org/article.aspx?articleid=21452
29. 28. K. Potter, J. Kniss, R. Riesenfeld, and C. R. Johnson. 2010. Visualizing summary statistics and uncertainty. Computer Graphics Forum 29, 3: 823-832. http://doi.org/10.1111/j.1467--8659.2009.01677.x
30. 29. Robert A Rigby and D Mikis Stasinopoulos. 2006. Using the Box-Cox t distribution in GAMLSS to model skewness and kurtosis. Statistical Modelling 6, 3: 209-229. Retrieved September 25, 2015 from http://smj.sagepub.com/content/6/3/209.abstract
31. 30. Michael Smithson and Jay Verkuilen. 2006. A better lemon squeezer? Maximum-likelihood regression with beta-distributed dependent variables. Psychological methods 11, 1: 54-71. Retrieved August 27, 2015 from http://www.ncbi.nlm.nih.gov/pubmed/16594767
32. 31. David J Spiegelhalter. 1999. Surgical Audit: Statistical Lessons from Nightingale and Codman. Journal of the Royal Statistical Society. Series A (Statistics in Society) 162: 45-58. http://doi.org/10.1111/1467--985X.00120
33. 32. Stan Development Team. 2015. Stan Modeling Language: User's Guide and Reference Manual.
34. 33. Barry N. Taylor and Chris E. Kuyatt. 1994. Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results.
35. 34. Edward R Tufte. 2006. Beautiful Evidence.
36. 35. Amos Tversky and Daniel Kahneman. 1975. Judgment under uncertainty: Heuristics and biases. Science 185, 4157: 1124-1131. Retrieved June 20, 2013 from http://link.springer.com/chapter/10.1007/978--94-0101834-0_8
37. 36. Dennis Vrecko, Alexander Klos, and Thomas Langer. 2009. Impact of Presentation Format and Self-Reported Risk Aversion on Revealed Skewness Preferences. Decision Analysis 6, 2: 57-74. Retrieved September 25, 2015 from http://dl.acm.org/citation.cfm?id=1555872.1555874
38. 37. Leland Wilkinson. 2012. Dot Plots. The American Statistician. Retrieved September 25, 2015 from http://amstat.tandfonline.com/doi/abs/10.1080/0003130 5.1999.10474474

Index Terms

• Human-centered computing