Abstract
Multisensorial books enrich a story with either traditional multimedia content or sensorial effects. The main idea is to increase children’s interest in reading by enhancing their QoE while reading. Studies on enriched and/or augmented e-books also propose synchronizing additional content with text. However, they usually focus on audio, vídeo, images, or haptic feedback. In this work, we present MBook, a tool for presenting multisensorial books. It decouples the book’s textual content from the additional content, as well as its synchronization, and rendering. Thus, a change in the additional content or its synchronization does not require changes to the book’s content. To enable fine-grained synchronization, MBook captures the reading position using an eye-tracker. Experimental results with students within the 13- to 19-year-old age group point to MBook being able to provide good usability.
- [1] . 2013. Information recall task impact in olfaction-enhanced multimedia. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 9, 3 (2013), 1–16. Google Scholar
Digital Library
- [2] . 2013. Mobile haptic e-book system to support 3D immersive reading in ubiquitous environments. ACM Transactions on Multimedia Computing, Communications, and Applications 9, 4 (2013), 27. Google Scholar
Digital Library
- [3] . 2011. Developing story-time: The importance of interactivity in encouraging childhood reading. Mousaion 29, 3 (2011), 193–208.Google Scholar
- [4] . 2019. Tracking the progression of reading through eye-gaze measurements. In 22th International Conference on Information Fusion (FUSION’19). 1–8.Google Scholar
- [5] . 1996. SUS-A quick and dirty usability scale. Usability Evaluation in Industry 189, 194 (1996), 4–7.Google Scholar
- [6] . 2016. A multimodal gaze-controlled virtual keyboard. IEEE Transactions on Human-Machine Systems 46, 4 (2016), 601–606.Google Scholar
Cross Ref
- [7] . 2009. EGameFlow: A scale to measure learners’ enjoyment of e-learning games. Computers & Education 52, 1 (2009), 101–112. Google Scholar
Digital Library
- [8] . 2014. Mulsemedia: State of the art, perspectives, and challenges. ACM Transactions on Multimedia Computing, Communications, and Applications 11, 1s (2014), 17. Google Scholar
Digital Library
- [9] . 2018. Evaluation of a visual–tactile multimodal display for surface obstacle avoidance during walking. IEEE Transactions on Human-Machine Systems 48, 6 (2018), 604–613.Google Scholar
Cross Ref
- [10] . 2017. Analysis of children’s interest in books and their reading levels depending on the education status of family. Quality & Quantity 52 (2017), 235–245.
DOI: https://doi.org/10.1007/s11135-017-0608-2Google Scholar - [11] . 2015. An augmented e-reader for multimodal literacy. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers. 353–356. Google Scholar
Digital Library
- [12] . 2018. Multisensorial books: Improving readers’ quality of experience. In 2018 XIII Latin American Conference on Learning Objects and Technology (LACLO’18).Google Scholar
Cross Ref
- [13] . 2015. Audiovisual semantic congruency during encoding enhances memory performance. Experimental Psychology 62, 2 (2015), 123–130. https://doi.org/10.1027/1618-3169/a000279Google Scholar
Cross Ref
- [14] . 2015. Application of immersive technologies for education: State of the art. In International Conference on Interactive Mobile Communication Technologies and Learning (IMCL’15). 283–288. https://doi.org/10.1109/IMCTL.2015.7359604Google Scholar
Cross Ref
- [15] . 2016. Supporting physical agents in an interactive e-book. In Genetic and Evolutionary Computing. Springer International Publishing, 243–252. Google Scholar
- [16] . 2017. A tutorial for olfaction-based multisensorial media application design and evaluation. ACM Computing Surveys 50, 5, Article
67 (Sept. 2017), 30 pages. https://doi.org/10.1145/3108243 Google ScholarDigital Library
- [17] . 2018. Designing a smart reading environment with and for children. In Proceedings of the 12th International Conference on Tangible, Embedded, and Embodied Interaction. 88–93. Google Scholar
Digital Library
- [18] . 2017. STREEN: Designing smart environments for story reading with children. International Journal on Interaction Design & Architecture(s) 32 (2017), 84–103.Google Scholar
- [19] . 2016. Embodied reading: A multisensory experience. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 1459–1466. Google Scholar
Digital Library
- [20] . 2016. Enhanced eBooks in the teaching/learning process of electronics. In 2nd International Conference on Higher Education Advances. 84–91.Google Scholar
- [21] . 2008. Benefits of multisensory learning. Trends in Cognitive Sciences 12, 11 (2008), 411–417. https://doi.org/10.1016/j.tics.2008.07.006Google Scholar
Cross Ref
- [22] . 2020. The multisensory experience of handling and reading books. Multisensory Research 33, 8 (2020), 902–928.
DOI: https://doi.org/10.1163/22134808-bja10015Google ScholarCross Ref
- [23] . 2019. Mulsemedia in telecommunication and networking education: A novel teaching approach that improves the learning process. IEEE Communications Magazine 57, 11 (2019), 60–66.Google Scholar
Digital Library
- [24] . 2014. Multiparty interaction understanding using smart multimodal digital signage. IEEE Transactions on Human-Machine Systems 44, 5 (2014), 625–637.Google Scholar
Cross Ref
- [25] . 2018. Evaluating the influence of mulsemedia content in reading. In Proceedings of the 24th Brazilian Symposium on Multimedia and the Web.
ACM , 133–136. Google ScholarDigital Library
- [26] . 2015. Beyond multimedia adaptation: Quality of experience-aware multi-sensorial media delivery. IEEE Transactions on Multimedia 17, 1 (2015), 104–117.Google Scholar
Cross Ref
- [27] . 2017. Can multisensorial media improve learner experience? In Proceedings of the 8th ACM on Multimedia Systems Conference. 315–320.Google Scholar
Digital Library
Index Terms
Using Multisensory Content to Impact the Quality of Experience of Reading Digital Books
Recommendations
Comparison of the Features of EPUB E-Book and SCORM E-Learning Content Model
E-books nowadays have greatly evolved in its presentation and functions, however its features for education need to be investigated and inspired because people who are accustomed to using printed books may consider and approach it in the same way as ...
Evaluating the Influence of Mulsemedia Content in Reading
WebMedia '18: Proceedings of the 24th Brazilian Symposium on Multimedia and the WebAugmented e-books provide the reader with a better quality of experience (QoE) by enriching the e-books with multimedia content and interactive elements. In the multimedia field, an improve in QoE has been performed by the use of sensory effects, ...
Evaluating reading fluency behavior via reading rates of elementary school students reading e-books
AbstractAs technology advances, paper-based text is transitioning to e-books, and this affects every aspect of educational practices. Fostering student literacy is critical to success. Reading ability is an important skill that allows young ...
Highlights- The ebook system record students' reading process for diagnosing reading fluency.






Comments