Abstract
Usability and accessibility guidelines aim to make graphical user interfaces accessible to all users, by, say, requiring that text is sufficiently large, interactive controls are visible, and heading size corresponds to importance. These guidelines must hold on the infinitely many possible renderings of a web page generated by differing screen sizes, fonts, and other user preferences. Today, these guidelines are tested by manual inspection of a few renderings, because 1) the guidelines are not expressed in a formal language, 2) the semantics of browser rendering are not well understood, and 3) no tools exist to check all possible renderings of a web page. VizAssert solves these problems. First, it introduces visual logic to precisely specify accessibility properties. Second, it formalizes a large fragment of the browser rendering algorithm using novel finitization reductions. Third, it provides a sound, automated tool for verifying assertions in visual logic.
We encoded 14 assertions drawn from best-practice accessibility and mobile-usability guidelines in visual logic. VizAssert checked them on on 62 professionally designed web pages. It found 64 distinct errors in the web pages, while reporting only 13 false positive warnings.
Supplemental Material
Available for Download
Appendices, including details on the interpretation of visual logic, full descriptions of each tested assertion and their formalization, and reproductions of the CSS standard's descriptions of line height, margin collapsing, and floating layout.
- Apple. 2017. Human Interface Guidelines. https://developer.apple. com/ios/human-interface-guidelines/Google Scholar
- Apple. 2017. UI Design Do’s and Don’ts. https://developer.apple.com/ design/tips/Google Scholar
- Apple Developer. 2017. UIKit Framework. https://developer.apple. com/documentation/uikitGoogle Scholar
- Greg J. Badros, Alan Borning, Kim Marriott, and Peter J. Stuckey. 1999. Constraint Cascading Style Sheets for the Web. In Proceedings of the 12th Annual ACM Symposium on User Interface Software and Technology (UIST’15). ACM, 73–82. Google Scholar
Digital Library
- Jeffrey P. Bigham. 2014. Making the Web Easier to See with Opportunistic Accessibility Improvement. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST ’14). ACM, 117–122. Google Scholar
Digital Library
- Alan Borning, Richard Lin, and Kim Marriott. 1997. Constraints for the Web. In Proceedings of the Fifth ACM International Conference on Multimedia (MULTIMEDIA ’97). ACM, 173–182. Google Scholar
Digital Library
- Matthew Butterick. 2010. Practical Typography. Matthew Butterick Typography, online only.Google Scholar
- Lyndon Cerejo. 2011. A User-Centered Approach to Web Design for Mobile Devices. https://www.smashingmagazine.com/2011/05/ a-user-centered-approach-to-web-design-for-mobile-devicesGoogle Scholar
- Tsung-Hsiang Chang, Tom Yeh, and Rob Miller. 2011. Associating the Visual Representation of User Interfaces with Their Internal Structures and Metadata. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). ACM, 245–256. Google Scholar
Digital Library
- Tsung-Hsiang Chang, Tom Yeh, and Robert C. Miller. 2010. GUI Testing Using Computer Vision. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10). ACM, 1535–1544. Google Scholar
Digital Library
- S. R. Choudhary, M. R. Prasad, and A. Orso. 2012. CrossCheck: Combining Crawling and Differencing to Better Detect Cross-browser Incompatibilities in Web Applications. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation. 171–180. Google Scholar
Digital Library
- Ravi Chugh, Brian Hempel, Mitchell Spradlin, and Jacob Albers. 2016. Programmatic and Direct Manipulation, Together at Last. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’16). ACM, 341–354. Google Scholar
Digital Library
- Burns David. 2012. Selenium 2 Testing Tools: Beginner’s Guide. Packt Publishing, Birmingham, UK. Google Scholar
Digital Library
- Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’08/ETAPS’08). Springer-Verlag, 337–340. http://dl.acm.org/citation.cfm?id=1792734.1792766 Google Scholar
Digital Library
- Diet. 2000. Basic Act on the Formation of an Advanced Information and Telecommunications Network Society. http://japan.kantei.go.jp/ it/it_basiclaw/it_basiclaw.htmlGoogle Scholar
- European Commission. 2016. Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies. http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv: OJ.L_.2016.327.01.0001.01.ENG&toc=OJ:L:2016:327:TOCGoogle Scholar
- Barry Feigenbaum and Michael Squillace. 2006. Accessibility Validation with RAVEN. In Proceedings of the 2006 International Workshop on Software Quality (WoSQ ’06). ACM, 27–32. Google Scholar
Digital Library
- LLC Formstack. 2014. Free Section 508 Compliance Checker. http: //www.508checker.com/checkGoogle Scholar
- Amy Fowler. 2017. A Swing Architecture Overview. http://www. oracle.com/technetwork/java/architecture-142923.htmlGoogle Scholar
- Free Website Templates. 2017. Free Website Templates. https:// freewebsitetemplates.comGoogle Scholar
- Sylvain Hallé, Nicolas Bergeron, Francis Guerin, and Gabriel Le Breton. 2015. Testing Web Applications Through Layout Constraints. In Software Testing, Verification and Validation (ICST), 2015 IEEE 8th International Conference on. IEEE, IEEE, 1–8.Google Scholar
- Osamu Hashimoto and Brad A. Myers. 1992. Graphical Styles for Building Interfaces by Demonstration. In Proceedings of the 5th Annual ACM Symposium on User Interface Software and Technology (UIST ’92). ACM, 117–124. Google Scholar
Digital Library
- Melody Ivory and Aline Chevalier. 2002. A Study of Automated Web Site Evaluation Tools. Technical Report. University of Washington, Department of Computer Science.Google Scholar
- Ranjitha Kumar, Jerry O. Talton, Salman Ahmad, and Scott R. Klemmer. 2011. Bricolage: Example-based Retargeting for Web Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, 2197–2206. Google Scholar
Digital Library
- Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2015. Automated Generation of Visual Web Tests from DOM-based Web Tests. In Proceedings of the 30th Annual ACM Symposium on Applied Computing (SAC ’15). ACM, 775–782. Google Scholar
Digital Library
- Hsiang-Sheng Liang, Kuan-Hung Kuo, Po-Wei Lee, Yu-Chien Chan, Yu-Chin Lin, and Mike Y. Chen. 2013. SeeSS: Seeing What I Broke – Visualizing Change Impact of Cascading Style Sheets (CSS). In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ’13). ACM, 353–356. Google Scholar
Digital Library
- Sonal Mahajan, Abdulmajeed Alameer, Phil McMinn, and William G. J. Halfond. 2017. Automated Repair of Layout Cross Browser Issues Using Search-based Techniques. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2017). ACM, 249–260. Google Scholar
Digital Library
- Jennifer Mankoff, Holly Fait, and Tu Tran. 2005. Is Your Web Page Accessible?: A Comparative Study of Methods for Assessing Web Page Accessibility for the Blind. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’05). ACM, 41–50. Google Scholar
Digital Library
- Atif Memon, Ishan Banerjee, and Adithya Nagarajan. 2003. GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing. In Proceedings of the 10th Working Conference on Reverse Engineering (WCRE ’03). IEEE Computer Society, 260–. http://dl.acm.org/citation. cfm?id=950792.951350 Google Scholar
Digital Library
- A. Mesbah and M. R. Prasad. 2011. Automated cross-browser compatibility testing. In 2011 33rd International Conference on Software Engineering (ICSE). 561–570. Google Scholar
Digital Library
- Leo A. Meyerovich and Rastislav Bodik. 2010. Fast and Parallel Webpage Layout. In Proceedings of the 19th International Conference on World Wide Web (WWW ’10). ACM, 711–720. Google Scholar
Digital Library
- Mozilla Developer Network. 2017. float. https://developer.mozilla.org/ en-US/docs/Web/CSS/floatGoogle Scholar
- Mozilla Developer Network. 2017. Mobile accessibility checklist. https://developer.mozilla.org/en-US/docs/Web/Accessibility/ Mobile_accessibility_checklistGoogle Scholar
- National Federation for the Blind. 2016. Blindness Statistics. https: //nfb.org/blindness-statisticsGoogle Scholar
- Tuan A. Nguyen and Christoph Csallner. 2015. Reverse engineering mobile application user interfaces with REMAUI. In Proc. 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) (ASE’15). IEEE, 248–259.Google Scholar
- Thomas Ostrand, Aaron Anodide, Herbert Foster, and Tarak Goradia. 1998. A Visual Test Development Environment for GUI Systems. In Proceedings of the 1998 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’98). ACM, 82–92. Google Scholar
Digital Library
- Jason Pamental. 2014. A More Modern Scale for Web Typography.Google Scholar
- Parasoft. 2017. Web UI Testing. https://www.parasoft.com/capability/ web-ui-testing/Google Scholar
- Pearson. 2017. Making E-Learning Accessible. http://wps.pearsoned. com/accessibility/Google Scholar
- Android Open Source Project. 2017. UI Overview. https://developer. android.com/guide/topics/ui/overview.htmlGoogle Scholar
- Murray Rowan, Peter Gregor, David Sloan, and Paul Booth. 2000. Evaluating Web Resources for Disability Access. In Proceedings of the Fourth International ACM Conference on Assistive Technologies (Assets ’00). ACM, 80–84. Google Scholar
Digital Library
- Shauvik Roy Choudhary, Husayn Versee, and Alessandro Orso. 2010. WEBDIFF: Automated Identification of Cross-browser Issues in Web Applications. In Proceedings of the 2010 IEEE International Conference on Software Maintenance (ICSM ’10). IEEE Computer Society, 1–10. Google Scholar
Digital Library
- Manolis Savva, Nicholas Kong, Arti Chhajta, Li Fei-Fei, Maneesh Agrawala, and Jeffrey Heer. 2011. ReVision: Automated Classification, Analysis and Redesign of Chart Images. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). ACM, 393–402. Google Scholar
Digital Library
- Terry Sullivan and Rebecca Matson. 2000. Barriers to Use: Usability and Content Accessibility on the Web’s Most Popular Sites. In Proceedings on the 2000 Conference on Universal Usability (CUU ’00). ACM, 139–144. Google Scholar
Digital Library
- Ivan E. Sutherland. 1964. Sketch Pad a Man-machine Graphical Communication System. In Proceedings of the SHARE Design Automation Workshop (DAC ’64). ACM, 6.329–6.346. Google Scholar
Digital Library
- Anthony T. 2012. Finger-friendly design: ideal mobile touchscreen target sizes. https://www.smashingmagazine.com/2012/02/ finger-friendly-design-ideal-mobile-touchscreen-target-sizes/Google Scholar
- US DOJ. 2010. Department of Justice Advanced Notice of Proposed Rulemaking, RIN 1190-AA61. https://www.ada.gov/anprm2010/web% 20anprm_2010.htmGoogle Scholar
- US DOJ. 2017. ADA Best Practices Tool Kit for State and Local Governments. https://www.ada.gov/pcatoolkit/chap5toolkit.htmGoogle Scholar
- Christopher J. van Wyk. 1982. A High-Level Language for Specifying Pictures. ACM Trans. Graph. 1, 2 (April 1982), 163–182. Google Scholar
Digital Library
- Minh N. Vu and Susan Ryan. {n. d.}. 2017 Website Accessibility Lawsuit Recap: A Tough Year for Businesses. https://www.adatitleiii.com/2018/01/ 2017-website-accessibility-lawsuit-recap-a-tough-year-for-businesses/Google Scholar
- W3C. 2008. Web Content Accessibility Guidelines 2.0. https://www. w3.org/TR/WCAG/Google Scholar
- W3C. 2011. Cascading Style Sheets Level 2 Revision 1 (CSS 2.1) Specification. https://www.w3.org/TR/2011/REC-CSS2-20110607/Google Scholar
- W3C. 2016. WAI-ARIA Overview. https://www.w3.org/WAI/intro/ariaGoogle Scholar
- W3Schools. 2017. CSS Dropdowns. https://www.w3schools.com/css/ css_dropdowns.aspGoogle Scholar
- Thomas A. Walsh, Gregory M. Kapfhammer, and Phil McMinn. 2017. Automated Layout Failure Detection for Responsive Web Pages Without an Explicit Oracle. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2017). ACM, 192–202. Google Scholar
Digital Library
- T. A. Walsh, P. McMinn, and G. M. Kapfhammer. 2015. Automatic Detection of Potential Layout Faults Following Changes to Responsive Web Pages (N). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). 709–714.Google Scholar
- Web Platform Tests. 2017. web-platform-tests dashboard, WPT/css/CSS2/floats. https://wptdashboard.appspot.com/css/ CSS2/floatsGoogle Scholar
- WebAIM. 2017. WAVE Web Accessibility Tool. http://wave.webaim. org/Google Scholar
- Hadley Wickham. 2009. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, New York, New York, USA. http://ggplot2. org Google Scholar
Digital Library
- Leland Wilkinson. 2005. The Grammar of Graphics (Statistics and Computing). Springer-Verlag New York, Inc., Secaucus, NJ, USA. Google Scholar
Digital Library
- Qing Xie and Atif M. Memon. 2007. Designing and Comparing Automated Test Oracles for GUI-based Software Applications. ACM Trans. Softw. Eng. Methodol. 16, 1, Article 4 (Feb. 2007), 38 pages. Google Scholar
Digital Library
- Brad Vander Zanden and Brad A. Myers. 1991. The Lapidary Graphical Interface Design Tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’91). ACM, 465–466. Google Scholar
Digital Library
Index Terms
Verifying that web pages have accessible layout
Recommendations
Verifying that web pages have accessible layout
PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and ImplementationUsability and accessibility guidelines aim to make graphical user interfaces accessible to all users, by, say, requiring that text is sufficiently large, interactive controls are visible, and heading size corresponds to importance. These guidelines must ...
Modular verification of web page layout
Automated verification can ensure that a web page satisfies accessibility, usability, and design properties regardless of the end user's device, preferences, and assistive technologies. However, state-of-the-art verification tools for layout properties ...
Designing web accessibility
The majority of websites have not been designed to make them accessible to all the people for whom they are intended. Nowadays, eEurope initiative (European Commission-eEurope, 2005), wants the institutions to become aware of the necessity of ...







Comments