Is your web page accessible?: a comparative study of methods for assessing web page accessibility for the blind

ABSTRACT

Web access for users with disabilities is an important goal and challenging problem for web content developers and designers. This paper presents a comparison of different methods for finding accessibility problems affecting users who are blind. Our comparison focuses on techniques that might be of use to Web developers without accessibility experience, a large and important group that represents a major source of inaccessible pages. We compare a laboratory study with blind users to an automated tool, expert review by web designers with and without a screen reader, and remote testing by blind users. Multiple developers, using a screen reader, were most consistently successful at finding most classes of problems, and tended to find about 50% of known problems. Surprisingly, a remote study with blind users was one of the least effective methods. All of the techniques, however, had different, complementary strengths and weaknesses.

References

1. Bobby Worldwide, http://bobby.watchfire.com/bobby/html/en/index.jsp]]
2. Bureau of the Census, "Survey of Income and Program Participation," 1994--95, http://www.census.go v/hhes/www/disable/dissipp.html]]
3. Clark, J., Building accessible websites, New Riders, IN, 2002.]]
4. Cockton, G., Lavery, D. and Woolrych, A., "Inspection-based evaluations," The Human Computer-Interaction Handbook, Jacko, J. A. and Sears, A. (editors), pp. 1118--1138.]]
5. Cockton, G. and Woolrych, A., "Understanding inspection methods: Lessons from an assessment of heuristic evaluation," In A. Blandford and J. Vanderdonckt, (Eds.), People & Computers XV, Springer-Verlag, pp. 171--192, 2001.]]
6. Colwell, C. and Petrie, H., "Evaluation of guidelines for designing accessible web content," In C. Bühler & H. Knops (Eds), Assistive technology on the threshold of the new millennium, IOS press, 1999.]]
7. Coyne, K. P. and Nielsen, J., "Beyond ALT Text: Making the web easy to use for users with disabilities," Nielsen, Norman Group, October, 2001, Available at: http://www.nngroup.com/reports/accessibility/]]
8. Coyne, K. P. and Nielsen, J., "How to conduct usability evaluations for accessibility: Methodology guidelines for testing websites and intranets with users who use assistive technology," Nielsen Norman Group, October, 2001, Available at: http://www.nngroup.com/reports/accessibility/testing/]]
9. Diaper, D. and Worman, L., "Two falls out of three in automated accessibility assessment of world wide web sites: A-prompt vs. Bobby," In Johnson, P. and Palanque, P. (Eds.), People and Computers XVII, Springer-Verlag.]]
10. Gray, W.D, and Salzman, M.C., "Damaged merchandise? A review of experiments that compare usability evaluation methods," Human-Computer Interaction, 13(3):203--261, 1998.]]
11. Hartson, H. R., Andre, T. S. and Williges, R. C., "Criteria for evaluating usability evaluation methods," International Journal of Human Computer Interaction, 13(4):373--410, 2001.]]
12. Hertzum, M. and Jacobsen, N. E., "The Evaluator Effect: A Chilling Fact about Usability Evaluation Methods," International Journal of Human Computer Interaction, 13(4):412--443, 2001.]]
13. Ivory, M. and Chevalier, A., "A Study of Automated Web Site Evaluation Tools," Technical Report UW-CSE-02-10-01, University of Washington, Department of Computer Science and Engineering, 2002.]]
14. LIFT, http://www.usablenet.com/]]
15. National Center for Health Statistics, "National health interview survey - disability supplement," 1994 and 1995.]]
16. Nielsen, J. and Molich, R., "Heuristic Evaluation of User Interfaces," In Proc. of CHI'90, pp. 249--256, ACM Press, 1990.]]
17. Paciello, M. G., Web accessibility for people with disabilities, CMP Books, KA, 2000.]]
18. Petrie, H. and Colwell, C., "Tool to assist authors in creating accessible web pages," In Proc. of NTEVH 98: Telematics in the education of the visually handicapped, 1998. http://www.snv.jussieu.fr/inova/publi/ntevh/tools.htm]]
19. Rowan, M., et al., "Evaluating web resources for disability access," In Proc. of ASSETS '00, pp. 80--84, ACM Press, 2000.]]
20. Sears, A., "Heuristic Walkthroughs: Finding the problems without the noise," International Journal of Human-Computer Interaction, 9:213--234.]]
21. Shneiderman, B. and Plaisant, C., Designing the user interface, 4th Edition, Pearson Education.]]
22. Sloan, D. et al., "Accessible accessibility," In Proc. of CUU'00, pp.96--101, ACM Press, 2000.]]
23. Sullivan, T. and Matson, R., "Barriers to use: Usability and content accessibility on the web's most popular sites," In Proc. of CUU'00, pp. 139--144, ACM Press, 2000.]]
24. Thatcher, J. et al., Accessible web sites, Springer-Verlag, NY, 2002.]]
25. Woolrych, A. and Cockton, G., "Assessing Heuristic Evaluation: Mind the quality, not just the percentages," in Proc. of HCI 2000, pp. 35--36, 2000.]]
26. W3C Markup Validation Service, http://validator.w3.org/]]
27. W3C Web Content Accessibility Guidelines 1.0, http://www.w3.org/TR/WAI-WEBCONTENT]]

Index Terms

• Applied computing

  • Document management and text processing

    • Document preparation

      • Multi / mixed media creation

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

    • Interaction paradigms

      • Hypertext / hypermedia

• Social and professional topics

  • Professional topics

    • Computing profession

      • Assistive technologies

  • User characteristics

    • People with disabilities