skip to main content
research-article
Public Access
Artifacts Available
Artifacts Evaluated & Functional

Verifying that web pages have accessible layout

Published:11 June 2018Publication History
Skip Abstract Section

Abstract

Usability and accessibility guidelines aim to make graphical user interfaces accessible to all users, by, say, requiring that text is sufficiently large, interactive controls are visible, and heading size corresponds to importance. These guidelines must hold on the infinitely many possible renderings of a web page generated by differing screen sizes, fonts, and other user preferences. Today, these guidelines are tested by manual inspection of a few renderings, because 1) the guidelines are not expressed in a formal language, 2) the semantics of browser rendering are not well understood, and 3) no tools exist to check all possible renderings of a web page. VizAssert solves these problems. First, it introduces visual logic to precisely specify accessibility properties. Second, it formalizes a large fragment of the browser rendering algorithm using novel finitization reductions. Third, it provides a sound, automated tool for verifying assertions in visual logic.

We encoded 14 assertions drawn from best-practice accessibility and mobile-usability guidelines in visual logic. VizAssert checked them on on 62 professionally designed web pages. It found 64 distinct errors in the web pages, while reporting only 13 false positive warnings.

Skip Supplemental Material Section

Supplemental Material

p1-panchekha.webm

References

  1. Apple. 2017. Human Interface Guidelines. https://developer.apple. com/ios/human-interface-guidelines/Google ScholarGoogle Scholar
  2. Apple. 2017. UI Design Do’s and Don’ts. https://developer.apple.com/ design/tips/Google ScholarGoogle Scholar
  3. Apple Developer. 2017. UIKit Framework. https://developer.apple. com/documentation/uikitGoogle ScholarGoogle Scholar
  4. Greg J. Badros, Alan Borning, Kim Marriott, and Peter J. Stuckey. 1999. Constraint Cascading Style Sheets for the Web. In Proceedings of the 12th Annual ACM Symposium on User Interface Software and Technology (UIST’15). ACM, 73–82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Jeffrey P. Bigham. 2014. Making the Web Easier to See with Opportunistic Accessibility Improvement. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST ’14). ACM, 117–122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Alan Borning, Richard Lin, and Kim Marriott. 1997. Constraints for the Web. In Proceedings of the Fifth ACM International Conference on Multimedia (MULTIMEDIA ’97). ACM, 173–182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Matthew Butterick. 2010. Practical Typography. Matthew Butterick Typography, online only.Google ScholarGoogle Scholar
  8. Lyndon Cerejo. 2011. A User-Centered Approach to Web Design for Mobile Devices. https://www.smashingmagazine.com/2011/05/ a-user-centered-approach-to-web-design-for-mobile-devicesGoogle ScholarGoogle Scholar
  9. Tsung-Hsiang Chang, Tom Yeh, and Rob Miller. 2011. Associating the Visual Representation of User Interfaces with Their Internal Structures and Metadata. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). ACM, 245–256. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Tsung-Hsiang Chang, Tom Yeh, and Robert C. Miller. 2010. GUI Testing Using Computer Vision. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10). ACM, 1535–1544. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. R. Choudhary, M. R. Prasad, and A. Orso. 2012. CrossCheck: Combining Crawling and Differencing to Better Detect Cross-browser Incompatibilities in Web Applications. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation. 171–180. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Ravi Chugh, Brian Hempel, Mitchell Spradlin, and Jacob Albers. 2016. Programmatic and Direct Manipulation, Together at Last. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’16). ACM, 341–354. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Burns David. 2012. Selenium 2 Testing Tools: Beginner’s Guide. Packt Publishing, Birmingham, UK. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’08/ETAPS’08). Springer-Verlag, 337–340. http://dl.acm.org/citation.cfm?id=1792734.1792766 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Diet. 2000. Basic Act on the Formation of an Advanced Information and Telecommunications Network Society. http://japan.kantei.go.jp/ it/it_basiclaw/it_basiclaw.htmlGoogle ScholarGoogle Scholar
  16. European Commission. 2016. Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies. http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv: OJ.L_.2016.327.01.0001.01.ENG&toc=OJ:L:2016:327:TOCGoogle ScholarGoogle Scholar
  17. Barry Feigenbaum and Michael Squillace. 2006. Accessibility Validation with RAVEN. In Proceedings of the 2006 International Workshop on Software Quality (WoSQ ’06). ACM, 27–32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. LLC Formstack. 2014. Free Section 508 Compliance Checker. http: //www.508checker.com/checkGoogle ScholarGoogle Scholar
  19. Amy Fowler. 2017. A Swing Architecture Overview. http://www. oracle.com/technetwork/java/architecture-142923.htmlGoogle ScholarGoogle Scholar
  20. Free Website Templates. 2017. Free Website Templates. https:// freewebsitetemplates.comGoogle ScholarGoogle Scholar
  21. Sylvain Hallé, Nicolas Bergeron, Francis Guerin, and Gabriel Le Breton. 2015. Testing Web Applications Through Layout Constraints. In Software Testing, Verification and Validation (ICST), 2015 IEEE 8th International Conference on. IEEE, IEEE, 1–8.Google ScholarGoogle Scholar
  22. Osamu Hashimoto and Brad A. Myers. 1992. Graphical Styles for Building Interfaces by Demonstration. In Proceedings of the 5th Annual ACM Symposium on User Interface Software and Technology (UIST ’92). ACM, 117–124. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Melody Ivory and Aline Chevalier. 2002. A Study of Automated Web Site Evaluation Tools. Technical Report. University of Washington, Department of Computer Science.Google ScholarGoogle Scholar
  24. Ranjitha Kumar, Jerry O. Talton, Salman Ahmad, and Scott R. Klemmer. 2011. Bricolage: Example-based Retargeting for Web Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, 2197–2206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2015. Automated Generation of Visual Web Tests from DOM-based Web Tests. In Proceedings of the 30th Annual ACM Symposium on Applied Computing (SAC ’15). ACM, 775–782. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Hsiang-Sheng Liang, Kuan-Hung Kuo, Po-Wei Lee, Yu-Chien Chan, Yu-Chin Lin, and Mike Y. Chen. 2013. SeeSS: Seeing What I Broke – Visualizing Change Impact of Cascading Style Sheets (CSS). In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ’13). ACM, 353–356. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Sonal Mahajan, Abdulmajeed Alameer, Phil McMinn, and William G. J. Halfond. 2017. Automated Repair of Layout Cross Browser Issues Using Search-based Techniques. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2017). ACM, 249–260. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Jennifer Mankoff, Holly Fait, and Tu Tran. 2005. Is Your Web Page Accessible?: A Comparative Study of Methods for Assessing Web Page Accessibility for the Blind. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’05). ACM, 41–50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Atif Memon, Ishan Banerjee, and Adithya Nagarajan. 2003. GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing. In Proceedings of the 10th Working Conference on Reverse Engineering (WCRE ’03). IEEE Computer Society, 260–. http://dl.acm.org/citation. cfm?id=950792.951350 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. A. Mesbah and M. R. Prasad. 2011. Automated cross-browser compatibility testing. In 2011 33rd International Conference on Software Engineering (ICSE). 561–570. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Leo A. Meyerovich and Rastislav Bodik. 2010. Fast and Parallel Webpage Layout. In Proceedings of the 19th International Conference on World Wide Web (WWW ’10). ACM, 711–720. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Mozilla Developer Network. 2017. float. https://developer.mozilla.org/ en-US/docs/Web/CSS/floatGoogle ScholarGoogle Scholar
  33. Mozilla Developer Network. 2017. Mobile accessibility checklist. https://developer.mozilla.org/en-US/docs/Web/Accessibility/ Mobile_accessibility_checklistGoogle ScholarGoogle Scholar
  34. National Federation for the Blind. 2016. Blindness Statistics. https: //nfb.org/blindness-statisticsGoogle ScholarGoogle Scholar
  35. Tuan A. Nguyen and Christoph Csallner. 2015. Reverse engineering mobile application user interfaces with REMAUI. In Proc. 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) (ASE’15). IEEE, 248–259.Google ScholarGoogle Scholar
  36. Thomas Ostrand, Aaron Anodide, Herbert Foster, and Tarak Goradia. 1998. A Visual Test Development Environment for GUI Systems. In Proceedings of the 1998 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’98). ACM, 82–92. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Jason Pamental. 2014. A More Modern Scale for Web Typography.Google ScholarGoogle Scholar
  38. Parasoft. 2017. Web UI Testing. https://www.parasoft.com/capability/ web-ui-testing/Google ScholarGoogle Scholar
  39. Pearson. 2017. Making E-Learning Accessible. http://wps.pearsoned. com/accessibility/Google ScholarGoogle Scholar
  40. Android Open Source Project. 2017. UI Overview. https://developer. android.com/guide/topics/ui/overview.htmlGoogle ScholarGoogle Scholar
  41. Murray Rowan, Peter Gregor, David Sloan, and Paul Booth. 2000. Evaluating Web Resources for Disability Access. In Proceedings of the Fourth International ACM Conference on Assistive Technologies (Assets ’00). ACM, 80–84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Shauvik Roy Choudhary, Husayn Versee, and Alessandro Orso. 2010. WEBDIFF: Automated Identification of Cross-browser Issues in Web Applications. In Proceedings of the 2010 IEEE International Conference on Software Maintenance (ICSM ’10). IEEE Computer Society, 1–10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Manolis Savva, Nicholas Kong, Arti Chhajta, Li Fei-Fei, Maneesh Agrawala, and Jeffrey Heer. 2011. ReVision: Automated Classification, Analysis and Redesign of Chart Images. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). ACM, 393–402. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Terry Sullivan and Rebecca Matson. 2000. Barriers to Use: Usability and Content Accessibility on the Web’s Most Popular Sites. In Proceedings on the 2000 Conference on Universal Usability (CUU ’00). ACM, 139–144. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Ivan E. Sutherland. 1964. Sketch Pad a Man-machine Graphical Communication System. In Proceedings of the SHARE Design Automation Workshop (DAC ’64). ACM, 6.329–6.346. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Anthony T. 2012. Finger-friendly design: ideal mobile touchscreen target sizes. https://www.smashingmagazine.com/2012/02/ finger-friendly-design-ideal-mobile-touchscreen-target-sizes/Google ScholarGoogle Scholar
  47. US DOJ. 2010. Department of Justice Advanced Notice of Proposed Rulemaking, RIN 1190-AA61. https://www.ada.gov/anprm2010/web% 20anprm_2010.htmGoogle ScholarGoogle Scholar
  48. US DOJ. 2017. ADA Best Practices Tool Kit for State and Local Governments. https://www.ada.gov/pcatoolkit/chap5toolkit.htmGoogle ScholarGoogle Scholar
  49. Christopher J. van Wyk. 1982. A High-Level Language for Specifying Pictures. ACM Trans. Graph. 1, 2 (April 1982), 163–182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Minh N. Vu and Susan Ryan. {n. d.}. 2017 Website Accessibility Lawsuit Recap: A Tough Year for Businesses. https://www.adatitleiii.com/2018/01/ 2017-website-accessibility-lawsuit-recap-a-tough-year-for-businesses/Google ScholarGoogle Scholar
  51. W3C. 2008. Web Content Accessibility Guidelines 2.0. https://www. w3.org/TR/WCAG/Google ScholarGoogle Scholar
  52. W3C. 2011. Cascading Style Sheets Level 2 Revision 1 (CSS 2.1) Specification. https://www.w3.org/TR/2011/REC-CSS2-20110607/Google ScholarGoogle Scholar
  53. W3C. 2016. WAI-ARIA Overview. https://www.w3.org/WAI/intro/ariaGoogle ScholarGoogle Scholar
  54. W3Schools. 2017. CSS Dropdowns. https://www.w3schools.com/css/ css_dropdowns.aspGoogle ScholarGoogle Scholar
  55. Thomas A. Walsh, Gregory M. Kapfhammer, and Phil McMinn. 2017. Automated Layout Failure Detection for Responsive Web Pages Without an Explicit Oracle. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2017). ACM, 192–202. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. T. A. Walsh, P. McMinn, and G. M. Kapfhammer. 2015. Automatic Detection of Potential Layout Faults Following Changes to Responsive Web Pages (N). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). 709–714.Google ScholarGoogle Scholar
  57. Web Platform Tests. 2017. web-platform-tests dashboard, WPT/css/CSS2/floats. https://wptdashboard.appspot.com/css/ CSS2/floatsGoogle ScholarGoogle Scholar
  58. WebAIM. 2017. WAVE Web Accessibility Tool. http://wave.webaim. org/Google ScholarGoogle Scholar
  59. Hadley Wickham. 2009. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, New York, New York, USA. http://ggplot2. org Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Leland Wilkinson. 2005. The Grammar of Graphics (Statistics and Computing). Springer-Verlag New York, Inc., Secaucus, NJ, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Qing Xie and Atif M. Memon. 2007. Designing and Comparing Automated Test Oracles for GUI-based Software Applications. ACM Trans. Softw. Eng. Methodol. 16, 1, Article 4 (Feb. 2007), 38 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Brad Vander Zanden and Brad A. Myers. 1991. The Lapidary Graphical Interface Design Tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’91). ACM, 465–466. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Verifying that web pages have accessible layout

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!