Abstract
Declarative multimedia documents represent the description of multimedia applications in terms of media items and relationships among them. Relationships specify how media items are dynamically arranged in time and space during runtime. Although a declarative approach usually facilitates the authoring task, authors can still make mistakes due to incorrect use of language constructs or inconsistent or missing relationships in a document. In order to properly support multimedia application authoring, it is important to provide tools with validation capabilities. Document validation can indicate possible inconsistencies in a given document to an author so that it can be revised before deployment. Although very useful, multimedia validation tools are not often provided by authoring tools.
This work proposes a multimedia validation approach that relies on a formal model called Simple Hypermedia Model (SHM). SHM is used for representing a document for the purpose of validation. An SHM document is validated using a hybrid approach based on two complementary techniques. The first one captures the document’s spatio-temporal layout in terms of its state throughout its execution by means of a rewrite theory, and validation is performed through model-checking. The second one captures the document’s layout in terms of intervals and event occurrences by means of Satisfiability Modulo Theories (SMT) formulas, and validation is performed through SMT solving. Due to different characteristics of both approaches, each validation technique complements the other in terms of expressiveness of SHM and tests to be checked.
We briefly present validation tools that use our approach. They were evaluated with real NCL documents and by usability tests.
- ABNT. 2011. Digital terrestrial television - Data coding and transmission specification for digital broadcasting - Part 2: Ginga-NCL for fixed and mobile receivers - XML application language for application coding. (2011). ABNT NBR 15606-2:2011 standard.Google Scholar
- J. F. Allen. 1983. Maintaining knowledge about temporal intervals. Communications of the ACM 26, 11 (1983), 832--843. Google Scholar
Digital Library
- Glauco F. Amorim, Joel A. F. dos Santos, and Débora C. Muchaluat-Saade. 2016. XTemplate 4.0: Providing adaptive layouts and nested templates for hypermedia documents. In Proceedings of the 22nd International Conference on Multimedia Modeling. Springer, Miami, FL, USA, 642--653. Google Scholar
Digital Library
- F. Barreto, D. Tamaki, Joel A. F. dos Santos, and D. C. Muchaluat-Saade. 2016. NCL-tester: Graphic application for NCL documents temporal test creation. In Proceedings of the 22th Brazilian Symposium on Multimedia and the Web (WebMedia’16). ACM, Teresina, Brazil, 91--94 {in Portuguese}. Google Scholar
Digital Library
- L. Belouaer and F. Maris. 2012. SMT spatio-temporal planning. In Proceedings of the ICAPS 2012 Workshop on Constraint Satisfaction Techniques for Planning and Scheduling Problems (COPLAS 2012). 6--15.Google Scholar
- Elisa Bertino, Elena Ferrari, Andrea Perego, and Diego Santi. 2005. A constraint-based approach for the authoring of multi-topic multimedia presentations. In Proceedings of the IEEE International Conference on Multimedia and Expo. IEEE Computer Society, Amsterdam, Netherlands, 578--581.Google Scholar
Cross Ref
- G. Blakowski and R. Steinmetz. 1996. A media synchronization survey: Reference model, specification and case studies. Journal on Selected Areas in Communications 14, 1 (January 1996), 5--35. Google Scholar
Digital Library
- Samia Bouyakoub and Abdelkader Belkhir. 2008. H-SMIL-net: A hierarchical petri net model for SMIL documents. In Proceedings of the Tenth International Conference on Computer Modeling and Simulation (UKSIM’08). IEEE Computer Society, Washington, DC, 106--111. Google Scholar
Digital Library
- Samia Bouyakoub and Abdelkader Belkhir. 2011. SMIL builder: An incremental authoring tool for SMIL documents. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) 7, 1 (Feb. 2011), 2:1--2:30. Google Scholar
Digital Library
- John Brooke. 1996. SUS-A quick and dirty usability scale. Usability Evaluation in Industry 189, 194 (1996), 4--7.Google Scholar
- M. Clavel, S. Eker, F. Durán, P. Lincoln, N. Martí-Oliet, and J. Meseguer. 2007. All About Maude - A High-performance Logical Framework: How to Specify, Program, and Verify Systems in Rewriting Logic. Vol. 4350. Springer-Verlag. Google Scholar
Digital Library
- Leonardo De Moura and Nikolaj Bjørner. 2011. Satisfiability modulo theories: Introduction and applications. Communications of the ACM 54, 9 (2011), 69--77. Google Scholar
Digital Library
- Joel A. F. dos Santos. 2016. Multimedia Document Validation Along Its Life Cycle. Ph.D. Dissertation. Universidade Federal Fluminense.Google Scholar
- Joel A. F. dos Santos, Christiano Braga, and Débora C Muchaluat-Saade. 2012. A model-driven approach for the analysis of multimedia document. In SLE (Doctoral Symposium). Dresden, Germany, 37--44.Google Scholar
- Joel A. F. dos Santos, Christiano Braga, and Débora C Muchaluat-Saade. 2013. An executable semantics for a multimedia authoring language. In Formal Methods: Foundations and Applications, Juliano Iyoda and Leonardo de Moura (Eds.), Vol. 8195. Springer, Brasília, Brazil, 67--82.Google Scholar
- Joel A. F. dos Santos, Christiano Braga, and Débora C. Muchaluat-Saade. 2015a. A rewriting logic semantics for NCL. Science of Computer Programming 107--108 (2015), 64--92. Google Scholar
Digital Library
- Joel A. F. dos Santos, Christiano Braga, Débora C. Muchaluat-Saade, Cécile Roisin, and Nabil Layaïda. 2015b. Spatio-temporal validation of multimedia documents. In Proceedings of the 2015 ACM Symposium on Document Engineering. ACM, Lausanne, Switzerland, 133--142. Google Scholar
Digital Library
- Joel A. F. dos Santos, Julia V. Silva, Renan R. Vasconcelos, Wagner Schau, Cláudia Werner, and Débora C. Muchaluat-Saade. 2012. aNa: API for NCL authoring. In Proceedings of the 18th Brazilian Symposium on Multimedia and the Web - Workshop of Tools and Applications. ACM, São Paulo, Brazil.Google Scholar
- Bruno Dutertre. 2014. Yices 2.2. In Computer-Aided Verification (CAV’2014) (Lecture Notes in Computer Science), Armin Biere and Roderick Bloem (Eds.), Vol. 8559. Springer, 737--744. Google Scholar
Digital Library
- S. Elias, K. S. Easwarakumar, and R. Chbeir. 2006. Dynamic consistency checking for temporal and spatial relations in multimedia presentations. In Proceedings of the 2006 ACM Symposium on Applied Computing. ACM, Dijon, France, 1380--1384. Google Scholar
Digital Library
- M. F. Felix. 2004. Formal Analysis of Software Models Oriented by Architectural Abstractions. Ph.D. Dissertation. Pontifícia Universidade Católica do Rio de Janeiro. in Portuguese.Google Scholar
- Ombretta Gaggi and Annalisa Bossi. 2011. Analysis and verification of SMIL documents. Multimedia Systems 17, 6 (April 2011), 487--506. Google Scholar
Digital Library
- Thomas R. G. Green and Marian Petre. 1996. Usability analysis of visual programming environments: A ‘cognitive dimensions’ framework. Journal of Visual Languages 8 Computing 7, 2 (1996), 131--174.Google Scholar
Cross Ref
- Hazel Lynda Hardman. 1998. Modeling and Authoring Hypermedia Documents. Ph.D. Dissertation. Universität Amsterdam. Retrieved from https://homepages.cwi.nl/lynda/thesis/.Google Scholar
- Roberto Ierusalimschy. 2006. Programming in Lua (2nd ed.). Roberto Ierusalimschy. Google Scholar
Digital Library
- ITU. 2009. Nested Context Language (NCL) and Ginga-NCL for IPTV services. http://www.itu.int/rec/T-REC-H.761-200904-S. (2009). ITU-T Recommendation H.761.Google Scholar
- M. Jourdan, N. Layaida, C. Roisin, L. Sabry-Ismail, and L. Tardif. 1998. Madeus, an authoring environment for interactive multimedia documents. In Proceedings of the 6th ACM International Conference on Multimedia. ACM, Bristol, England, 267--272. Google Scholar
Digital Library
- Delcino Picinin Júnior, Jean-Marie Farines, and Cristian Koliver. 2012. An approach to verify live NCL applications. In Proceedings of the 18th Brazilian Symposium on Multimedia and the Web. ACM, Salvador, Brazil, 223--232. Google Scholar
Digital Library
- Ioannis Kostalas, T. Sellis, and Michalis Vazirgiannis. 1999. Spatiotemporal specification 8 verification of multimedia scenarios. In Database Semantics, Zahir Tari and Scott Stevens (Eds.), Vol. 11. Springer, 169--188. Google Scholar
Digital Library
- Sébastien Laborie, Jérôme Euzenat, and Nabil Layaïda. 2011. Semantic adaptation of multimedia documents. Multimedia Tools and Applications 55, 3 (2011), 379--398. Google Scholar
Digital Library
- Guilherme Augusto Ferreira Lima and Luiz Fernando Gomes Soares. 2013. Two normal forms for link-connector pairs in NCL 3.0. In Proceedings of the 19th Brazilian Symposium on Multimedia and the Web. ACM, Salvador, Brazil, 201--204. Google Scholar
Digital Library
- Huadong Ma and K. G. Shin. 2004. Checking consistency in multimedia synchronization constraints. IEEE Transactions on Multimedia 6, 4 (Aug 2004), 565--574. Google Scholar
Digital Library
- J. Meseguer. 2012. Twenty years of rewriting logic. The Journal of Logic and Algebraic Programming 81, 7 (2012), 721--781.Google Scholar
Cross Ref
- A. Pnueli. 1977. The temporal logic of programs. In 18th Annual Symposium on Foundations of Computer Science. IEEE Computer Society, Washington, DC, USA, 46--57. Google Scholar
Digital Library
- David A. Randell, Zhan Cui, and Anthony G. Cohn. 1992. A spatial logic based on regions and connection. Principles of Knowledge Representation and Reasoning (1992), 165--176. Google Scholar
Digital Library
- L. F. G. Soares and R. F. Rodrigues. 2005. Nested Context Model 3.0 Part 1 - NCM Core. Technical Report. Informatics Department, PUC-Rio, Rio de Janeiro.Google Scholar
- Luiz Fernando G. Soares, Rogério Ferreira Rodrigues, Renato Cerqueira, and Simone Diniz Junqueira Barbosa. 2010. Variable and state handling in NCL. Multimedia Tools and Applications 50, 3 (2010), 465--489. Google Scholar
Digital Library
- W3C. 2008. Synchronized Multimedia Integration Language - SMIL 3.0 Specification. World-Wide Web Consortium Working Recommendation. Retrieved from http://www.w3c.org/TR/SMIL3.Google Scholar
- W3C. 2011. Scalable Vector Graphics (SVG) 1.1. World-Wide Web Consortium Working Recommendation Retrieved from http://www.w3.org/TR/SVG11.Google Scholar
- W3C. 2014. HTML5: A vocabulary and associated APIs for HTML and XHTML. World-Wide Web Consortium Candidate Recommendation. https://www.w3.org/TR/2014/REC-html5-20141028.Google Scholar
- W3C. 2014. Web Animations 1.0. World-Wide Web Consortium Working Draft. Retrieved from http://www.w3.org/TR/web-animations/.Google Scholar
Index Terms
A Hybrid Approach for Spatio-Temporal Validation of Declarative Multimedia Documents
Recommendations
Spatio-temporal Validation of Multimedia Documents
DocEng '15: Proceedings of the 2015 ACM Symposium on Document EngineeringA multimedia document authoring system should provide analysis and validation tools that help authors find and correct mistakes before document deployment. Although very useful, multimedia validation tools are not often provided. Spatial validation of ...
Challenges in Interactive Multimedia Document Consistency Verification
Webmedia '16: Proceedings of the 22nd Brazilian Symposium on Multimedia and the WebA multimedia document may be specified manually or automatically generated, instantiated, adapted or has its content and structure dynamically edited. Therefore, changes in a document specification may occur from its creation to its execution. Such ...
STEVE: a Hypermedia Authoring Tool based on the Simple Interactive Multimedia Model
DocEng '18: Proceedings of the ACM Symposium on Document Engineering 2018This paper proposes an interactive multimedia authoring tool called STEVE (Spatio-Temporal View Editor) and a new multimedia model called SIMM (Simple Interactive Multimedia Model). STEVE aims at allowing users with no knowledge of multimedia authoring ...






Comments