skip to main content
research-article

A Hybrid Approach for Spatio-Temporal Validation of Declarative Multimedia Documents

Published:30 October 2018Publication History
Skip Abstract Section

Abstract

Declarative multimedia documents represent the description of multimedia applications in terms of media items and relationships among them. Relationships specify how media items are dynamically arranged in time and space during runtime. Although a declarative approach usually facilitates the authoring task, authors can still make mistakes due to incorrect use of language constructs or inconsistent or missing relationships in a document. In order to properly support multimedia application authoring, it is important to provide tools with validation capabilities. Document validation can indicate possible inconsistencies in a given document to an author so that it can be revised before deployment. Although very useful, multimedia validation tools are not often provided by authoring tools.

This work proposes a multimedia validation approach that relies on a formal model called Simple Hypermedia Model (SHM). SHM is used for representing a document for the purpose of validation. An SHM document is validated using a hybrid approach based on two complementary techniques. The first one captures the document’s spatio-temporal layout in terms of its state throughout its execution by means of a rewrite theory, and validation is performed through model-checking. The second one captures the document’s layout in terms of intervals and event occurrences by means of Satisfiability Modulo Theories (SMT) formulas, and validation is performed through SMT solving. Due to different characteristics of both approaches, each validation technique complements the other in terms of expressiveness of SHM and tests to be checked.

We briefly present validation tools that use our approach. They were evaluated with real NCL documents and by usability tests.

References

  1. ABNT. 2011. Digital terrestrial television - Data coding and transmission specification for digital broadcasting - Part 2: Ginga-NCL for fixed and mobile receivers - XML application language for application coding. (2011). ABNT NBR 15606-2:2011 standard.Google ScholarGoogle Scholar
  2. J. F. Allen. 1983. Maintaining knowledge about temporal intervals. Communications of the ACM 26, 11 (1983), 832--843. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Glauco F. Amorim, Joel A. F. dos Santos, and Débora C. Muchaluat-Saade. 2016. XTemplate 4.0: Providing adaptive layouts and nested templates for hypermedia documents. In Proceedings of the 22nd International Conference on Multimedia Modeling. Springer, Miami, FL, USA, 642--653. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. F. Barreto, D. Tamaki, Joel A. F. dos Santos, and D. C. Muchaluat-Saade. 2016. NCL-tester: Graphic application for NCL documents temporal test creation. In Proceedings of the 22th Brazilian Symposium on Multimedia and the Web (WebMedia’16). ACM, Teresina, Brazil, 91--94 {in Portuguese}. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. L. Belouaer and F. Maris. 2012. SMT spatio-temporal planning. In Proceedings of the ICAPS 2012 Workshop on Constraint Satisfaction Techniques for Planning and Scheduling Problems (COPLAS 2012). 6--15.Google ScholarGoogle Scholar
  6. Elisa Bertino, Elena Ferrari, Andrea Perego, and Diego Santi. 2005. A constraint-based approach for the authoring of multi-topic multimedia presentations. In Proceedings of the IEEE International Conference on Multimedia and Expo. IEEE Computer Society, Amsterdam, Netherlands, 578--581.Google ScholarGoogle ScholarCross RefCross Ref
  7. G. Blakowski and R. Steinmetz. 1996. A media synchronization survey: Reference model, specification and case studies. Journal on Selected Areas in Communications 14, 1 (January 1996), 5--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Samia Bouyakoub and Abdelkader Belkhir. 2008. H-SMIL-net: A hierarchical petri net model for SMIL documents. In Proceedings of the Tenth International Conference on Computer Modeling and Simulation (UKSIM’08). IEEE Computer Society, Washington, DC, 106--111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Samia Bouyakoub and Abdelkader Belkhir. 2011. SMIL builder: An incremental authoring tool for SMIL documents. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) 7, 1 (Feb. 2011), 2:1--2:30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. John Brooke. 1996. SUS-A quick and dirty usability scale. Usability Evaluation in Industry 189, 194 (1996), 4--7.Google ScholarGoogle Scholar
  11. M. Clavel, S. Eker, F. Durán, P. Lincoln, N. Martí-Oliet, and J. Meseguer. 2007. All About Maude - A High-performance Logical Framework: How to Specify, Program, and Verify Systems in Rewriting Logic. Vol. 4350. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Leonardo De Moura and Nikolaj Bjørner. 2011. Satisfiability modulo theories: Introduction and applications. Communications of the ACM 54, 9 (2011), 69--77. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Joel A. F. dos Santos. 2016. Multimedia Document Validation Along Its Life Cycle. Ph.D. Dissertation. Universidade Federal Fluminense.Google ScholarGoogle Scholar
  14. Joel A. F. dos Santos, Christiano Braga, and Débora C Muchaluat-Saade. 2012. A model-driven approach for the analysis of multimedia document. In SLE (Doctoral Symposium). Dresden, Germany, 37--44.Google ScholarGoogle Scholar
  15. Joel A. F. dos Santos, Christiano Braga, and Débora C Muchaluat-Saade. 2013. An executable semantics for a multimedia authoring language. In Formal Methods: Foundations and Applications, Juliano Iyoda and Leonardo de Moura (Eds.), Vol. 8195. Springer, Brasília, Brazil, 67--82.Google ScholarGoogle Scholar
  16. Joel A. F. dos Santos, Christiano Braga, and Débora C. Muchaluat-Saade. 2015a. A rewriting logic semantics for NCL. Science of Computer Programming 107--108 (2015), 64--92. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Joel A. F. dos Santos, Christiano Braga, Débora C. Muchaluat-Saade, Cécile Roisin, and Nabil Layaïda. 2015b. Spatio-temporal validation of multimedia documents. In Proceedings of the 2015 ACM Symposium on Document Engineering. ACM, Lausanne, Switzerland, 133--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Joel A. F. dos Santos, Julia V. Silva, Renan R. Vasconcelos, Wagner Schau, Cláudia Werner, and Débora C. Muchaluat-Saade. 2012. aNa: API for NCL authoring. In Proceedings of the 18th Brazilian Symposium on Multimedia and the Web - Workshop of Tools and Applications. ACM, São Paulo, Brazil.Google ScholarGoogle Scholar
  19. Bruno Dutertre. 2014. Yices 2.2. In Computer-Aided Verification (CAV’2014) (Lecture Notes in Computer Science), Armin Biere and Roderick Bloem (Eds.), Vol. 8559. Springer, 737--744. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. S. Elias, K. S. Easwarakumar, and R. Chbeir. 2006. Dynamic consistency checking for temporal and spatial relations in multimedia presentations. In Proceedings of the 2006 ACM Symposium on Applied Computing. ACM, Dijon, France, 1380--1384. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. M. F. Felix. 2004. Formal Analysis of Software Models Oriented by Architectural Abstractions. Ph.D. Dissertation. Pontifícia Universidade Católica do Rio de Janeiro. in Portuguese.Google ScholarGoogle Scholar
  22. Ombretta Gaggi and Annalisa Bossi. 2011. Analysis and verification of SMIL documents. Multimedia Systems 17, 6 (April 2011), 487--506. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Thomas R. G. Green and Marian Petre. 1996. Usability analysis of visual programming environments: A ‘cognitive dimensions’ framework. Journal of Visual Languages 8 Computing 7, 2 (1996), 131--174.Google ScholarGoogle ScholarCross RefCross Ref
  24. Hazel Lynda Hardman. 1998. Modeling and Authoring Hypermedia Documents. Ph.D. Dissertation. Universität Amsterdam. Retrieved from https://homepages.cwi.nl/lynda/thesis/.Google ScholarGoogle Scholar
  25. Roberto Ierusalimschy. 2006. Programming in Lua (2nd ed.). Roberto Ierusalimschy. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. ITU. 2009. Nested Context Language (NCL) and Ginga-NCL for IPTV services. http://www.itu.int/rec/T-REC-H.761-200904-S. (2009). ITU-T Recommendation H.761.Google ScholarGoogle Scholar
  27. M. Jourdan, N. Layaida, C. Roisin, L. Sabry-Ismail, and L. Tardif. 1998. Madeus, an authoring environment for interactive multimedia documents. In Proceedings of the 6th ACM International Conference on Multimedia. ACM, Bristol, England, 267--272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Delcino Picinin Júnior, Jean-Marie Farines, and Cristian Koliver. 2012. An approach to verify live NCL applications. In Proceedings of the 18th Brazilian Symposium on Multimedia and the Web. ACM, Salvador, Brazil, 223--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Ioannis Kostalas, T. Sellis, and Michalis Vazirgiannis. 1999. Spatiotemporal specification 8 verification of multimedia scenarios. In Database Semantics, Zahir Tari and Scott Stevens (Eds.), Vol. 11. Springer, 169--188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Sébastien Laborie, Jérôme Euzenat, and Nabil Layaïda. 2011. Semantic adaptation of multimedia documents. Multimedia Tools and Applications 55, 3 (2011), 379--398. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Guilherme Augusto Ferreira Lima and Luiz Fernando Gomes Soares. 2013. Two normal forms for link-connector pairs in NCL 3.0. In Proceedings of the 19th Brazilian Symposium on Multimedia and the Web. ACM, Salvador, Brazil, 201--204. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Huadong Ma and K. G. Shin. 2004. Checking consistency in multimedia synchronization constraints. IEEE Transactions on Multimedia 6, 4 (Aug 2004), 565--574. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. J. Meseguer. 2012. Twenty years of rewriting logic. The Journal of Logic and Algebraic Programming 81, 7 (2012), 721--781.Google ScholarGoogle ScholarCross RefCross Ref
  34. A. Pnueli. 1977. The temporal logic of programs. In 18th Annual Symposium on Foundations of Computer Science. IEEE Computer Society, Washington, DC, USA, 46--57. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. David A. Randell, Zhan Cui, and Anthony G. Cohn. 1992. A spatial logic based on regions and connection. Principles of Knowledge Representation and Reasoning (1992), 165--176. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. L. F. G. Soares and R. F. Rodrigues. 2005. Nested Context Model 3.0 Part 1 - NCM Core. Technical Report. Informatics Department, PUC-Rio, Rio de Janeiro.Google ScholarGoogle Scholar
  37. Luiz Fernando G. Soares, Rogério Ferreira Rodrigues, Renato Cerqueira, and Simone Diniz Junqueira Barbosa. 2010. Variable and state handling in NCL. Multimedia Tools and Applications 50, 3 (2010), 465--489. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. W3C. 2008. Synchronized Multimedia Integration Language - SMIL 3.0 Specification. World-Wide Web Consortium Working Recommendation. Retrieved from http://www.w3c.org/TR/SMIL3.Google ScholarGoogle Scholar
  39. W3C. 2011. Scalable Vector Graphics (SVG) 1.1. World-Wide Web Consortium Working Recommendation Retrieved from http://www.w3.org/TR/SVG11.Google ScholarGoogle Scholar
  40. W3C. 2014. HTML5: A vocabulary and associated APIs for HTML and XHTML. World-Wide Web Consortium Candidate Recommendation. https://www.w3.org/TR/2014/REC-html5-20141028.Google ScholarGoogle Scholar
  41. W3C. 2014. Web Animations 1.0. World-Wide Web Consortium Working Draft. Retrieved from http://www.w3.org/TR/web-animations/.Google ScholarGoogle Scholar

Index Terms

  1. A Hybrid Approach for Spatio-Temporal Validation of Declarative Multimedia Documents

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!