Abstract
An increasing number of novel applications produce a rich set of different data types that need to be managed efficiently and coherently. In this article we present our experience with designing and implementing a data management infrastructure for a distributed immersive performance (DIP) application. The DIP project investigates a versatile framework for the capture, recording, and replay of video, audio, and MIDI (Musical Instrument Digital Interface) streams in an interactive environment for collaborative music performance. We are focusing on two classes of data streams that are generated within this environment. The first category consists of high-resolution isochronous media streams, namely audio and video. The second class comprises MIDI data produced by electronic instruments. MIDI event sequences are alphanumeric in nature and fall into the category of the data streams that have been of interest to data management researchers in recent years.
We present our data management architecture, which provides a repository for all DIP data. Streams of both categories need to be acquired, transmitted, stored, and replayed in real time. Data items are correlated across different streams with temporal indices. The audio and video streams are managed in our own High-performance Data Recording Architecture (HYDRA), which integrates multistream recording and retrieval in a consistent manner. This paper reports on the practical issues and challenges that we encountered during the design, implementation and experimental phases of our prototype. We also present some analysis results and discuss future extensions for the architecture.
- Blakowski, G. and Steinmetz, R. 1996. A media synchronization survey: Reference model, specification, and case studies. IEEE J. Select. Areas Comm. 14, 1, 5--35. Google Scholar
Digital Library
- Bolosky, W. J., Barrera, J. S., Draves, R. P., Fitzgerald, R. P., Gibson, G. A., Jones, M. B., Levi, S. P., Myhrvold, N. P., and Rashid, R. F. 1996. The tiger video fileserver. In Proceedings of the 6th Workshop on Network and Operating System Support for Digital Audio and Video. Zushi, Japan.Google Scholar
- CCRMA. 2002. Stanford University's SoundWire Group at CCRMA, jam session. www.ccrma.stanford.edu/groups/soundwire.Google Scholar
- Chafe, C. 2005. Low-latency audio, networked musical performance experiments. ccrma.stanford.edu/~cc.Google Scholar
- Chafe, C., Gurevich, M., Leslie, G., and Tyan, S. 2004. Effect of time delay on ensemble accuracy. In Proceedings of the International Symposium on Musical Acoustics (ISMA'04). Nara, Japan.Google Scholar
- Chafe, C., Wilson, S., Leistikow, R., Chisholm, D., and Scavone, G. 2000. A simplified approach to high quality music and sound over IP. In Proceedings of the COST G-6 Conference on Digital Audio Effects (DAFX'00). Verona, Italy.Google Scholar
- Chandrasekaran, S. 2003. Telegraphcq: Continuous dataflow processing for an uncertain world. In Proceedings of the 1st Biennial Conference on Innovative Data Systems Research (CIDR 2003). Asilomar, CA. Google Scholar
Digital Library
- Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005. Segmental tempo analysis of performances in performer-centered experiments in the distributed immersive performance project. In Proceedings of International Conference on Sound and Music Computing (SMC'05). Salerno, Italy.Google Scholar
- Chew, E., Sawchuk, A., Zimmermann, R., the Tosheff Piano Duo (Stoyanova, V., Tosheff, I.), Kyriakakis, C., Papadopoulos, C., François, A., and Volk, A. 2004. Distributed immersive performance. In Proceedings of the Annual National Association of the Schools of Music Meeting (NASM'04). San Diego, CA.Google Scholar
- Chew, E., Zimmermann, R., Sawchuk, A., Kyriakakis, C., Papadopoulos, C., François, A., Kim, G., and Volk, A. 2004. Musical interaction at a distance: Distributed immersive performance. In Proceedings of the Open Workshop of MUSICNETWORK: Integration of Music in Multimedia Applications (MUSICNETWORK'04). Barcelona, Spain.Google Scholar
- Chew, E., Zimmermann, R., Sawchuk, A., Papadopoulos, C., Kyriakakis, C., Tanoue, C., Desai, D., Pawar, M., Sinha, R., and Meyer, W. 2005. A second report on the user experiments in the distributed immersive performance project. In Proceedings of the Open Workshop of MUSICNETWORK: Integration of Music in Multimedia Applications (MUSICNETWORK'05). Vienna, Austria.Google Scholar
- Cooperstock, J. R. and Spackman, S. P. 2001. The recording studio that spanned a continent. In Proceedings of the IEEE International Conference on Web Delivering of Music (WEDELMUSIC'01). Florence, Italy. Google Scholar
Digital Library
- Ghandeharizadeh, S., Zimmermann, R., Shi, W., Rejaie, R., Ierardi, D., and Li, T. 1997. Mitra: A scalable continuous media server. Kluwer Multimedia Tools Appl. 5, 1, 79--108. Google Scholar
Digital Library
- Gresham-Lancaster, S. 2005. AB_Time Vancouver-Marseilles-Troy concert at NIME'05. Vancouver, Canada. hct.ece.ubc.ca/nime/2005/concerts.html.Google Scholar
- Gu, X., Dick, M., Kurtisi, Z., Noyer, U., and Wolf, L. 2005. Network-centric music performance: Practice and experiments. IEEE Comm. Mag. 43, 6, 86--93. Google Scholar
Digital Library
- Hsieh, J., Liu, J., Du, D., Ruwart, T., and Lin, M. 1995. Experimental performance of a mass storage system for video-on-demand. (Special Issue of Multimedia Systems and Technology) J. Parall. Distrib. Comput. (JPDC) 30, 2, 147--167. Google Scholar
Digital Library
- Internet2 Member Meeting. 2004. The University of Texas at Austin and Internet2 to Host first virtual concert experiment. Internet2 Member Meeting, Austin, TX. www.utexas.edu/opa/news/04newsreleases/nr_200409/nr_internet2_040927.html#info.Google Scholar
- Kanki, S. 1998. Mélange à trois. NOTAM. www.notam02.no/warsaw/melange.html.Google Scholar
- Konstantas, D. 1998. Overview of telepresence environment for distributed musical rehearsals. In Proceedings of the ACM Symposium on Applied Computing (SAC'98). Atlanta, GA. Google Scholar
Digital Library
- Lambrinos, L., Kirstein, P., and Hardman, V. 1998. The multicast multimedia conference recorder. In Proceedings of the International Conference on Computer Communications and Networks (IC3N). Lafayette, LA. Google Scholar
Digital Library
- Laursen, A., Olkin, J., and Porter, M. 1994. Oracle media server: Providing consumer based interactive access to multimedia data. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 470--477. Google Scholar
Digital Library
- Lazzaro, J. and Wawrzynek, J. 2001. A case for network musical performance. In Proceedings of the International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV'01). New York, NY. Google Scholar
Digital Library
- Lie, A. and Rønningen, L. A. 2003. Distributed multimedia plays with QoS guaranties over IP. In Proceedings of the International Conference on WEB Delivering of Music (WEDELMUSIC'03). Leeds, UK.Google Scholar
- Martin, C., Narayan, P. S., Özden, B., Rastogi, R., and Silberschatz, A. 1996. The fellini multimedia storage server. In Multimedia Information Storage and Management, S. M. Chung, Ed. Kluwer Academic Publishers, Boston, MA, Chapter 5.Google Scholar
- McLeod, D., Neumann, U., Nikias, C., and Sawchuk, A. 1999. Integrated media systems. IEEE Signal Process. Magazine 16, 33--76.Google Scholar
Cross Ref
- Mills, D. L. 1991. Internet time synchronization: The network time protocol. IEEE Trans. Comm. 38, 10, 1482--1493.Google Scholar
Cross Ref
- Muntz, R., Santos, J., and Berson, S. 1997. RIO: A real-time multimedia object server. ACM Sigmet. Perform. Eval. Rev. 25, 2, 29--35. Google Scholar
Digital Library
- Perrot, X. 2004. 2004 Berlin-Paris network concert at ICHIM'04. Berlin, Germany. www.ichim.org/jahia/Jahia/lang/en.Google Scholar
- PostgreSQL Group. 2003. PostgreSQL Open Source database system. The PostgreSQL Group. www.postgresql.org.Google Scholar
- Robbins, S. 1995. A MIDI primer for computer scientists. Division of Computer Science Tech. Rep. CS-95-11, University of Texas, San Antonio.Google Scholar
- Rosen, T. 2004. Is it live or is it Internet? Austin, Texas. www.utexas.edu/features/archive/2004/internet.html.Google Scholar
- Sawchuk, A. A., Chew, E., Zimmermann, R., Papadopoulos, C., and Kyriakakis, C. 2003. From remote media immersion to distributed immersive performance. In Proceedings of the ACM SIGMM 2003 Workshop on Experiential Telepresence (ETP 2003). Berkeley, CA. Google Scholar
Digital Library
- Schooler, E. 2001. Distributed music: A foray into networked performance, Haydn piano Trio No.1 in G, the finale movement. www.postel.org/pipermail/end2end-interest/2001-August/001314.html.Google Scholar
- Schulzrinne, H., Casner, S., Frederick, R., and Jacobson, V. 1996. RTP: A transport protocol for real time applications. www.ietf.org/rfc/rfc1889.txt.Google Scholar
- Schulzrinne, H., Rao, A., and Lanphier, R. 1998. Real time streaming protocol (RTSP). www.ietf.org/rfc/rfc2326.txt. Google Scholar
Digital Library
- Shahabi, C., Zimmermann, R., Fu, K., and Yao, S.-Y. D. 2002. Yima: A second generation continuous media server. IEEE Comput. 35, 6, 56--64. Google Scholar
Digital Library
- Tobagi, F., Pang, J., Baird, R., and Gang, M. 1993. Streaming RAID-A disk array management system for video files. In Proceedings of the 1st ACM International Conference on Multimedia. Anaheim, CA, 393--400. Google Scholar
Digital Library
- Witherspoon, J. T., Schuckman, L., and Spilker, J. 1978. Time transfer via GPS. In Proceedings of the Annual Precise Time and Time Interval Applications and Planning Meeting. Goddard Space Flight Center, Greenbelt, MD.Google Scholar
- Xu, A., Woszczyk, W., Settel, Z., Pennycook, B., Rowe, R., Galanter, P., Bary, J., Martin, G., Corey, J., and Cooperstock, J. R. 2000. Real-time streaming of multichannel audio data over Internet. J. Audio Eng. Soc. 48, 7--8.Google Scholar
- Young, J. P. and Fujinaga, I. 1999. Piano master classes via the Internet. In Proceedings of the International Computer Music Conference (ICMA'99). Beijing, China.Google Scholar
- Zimmermann, R., Fu, K., Jahangiri, M., and Shahabi, C. 2006. A multi-threshold online smoothing technique for variable rate multimedia streams. Multimedia Tools Appl. 28, 1. Google Scholar
Digital Library
- Zimmermann, R., Fu, K., and Ku, W.-S. 2003. Design of a large scale data stream recorder. In Proceedings of the 5th International Conference on Enterprise Information Systems (ICEIS'03). Angers, France. www.iceis.org.Google Scholar
- Zimmermann, R., Fu, K., Shahabi, C., Yao, S.-Y. D., and Zhu, H. 2001. Yima: Design and evaluation of a streaming media system for residential broadband services. In Proceedings of the VLDB Workshop on Databases in Telecommunications (DBTel'01). Rome, Italy. Google Scholar
Digital Library
- Zimmermann, R., Kyriakakis, C., Shahabi, C., Papadopoulos, C., Sawchuk, A. A., and Neumann, U. 2004. RMI system: Internet meets the future home theater. IEEE Multimedia (Special issue on “Digital Multimedia on Demand”) 11, 2, 48--57. Google Scholar
Digital Library
- Zimmermann, R., Shahabi, C., Fu, K., and Yao, S.-Y. D. 2004. Scalability evaluation of the Yima streaming media architecture. Softw. Pract. Exper. 35, 4, 345--359. Google Scholar
Digital Library
Index Terms
Distributed musical performances: Architecture and stream management
Recommendations
Segregating Musical Chords for Automatic Music Transcription: A LSTM-RNN Approach
Pattern Recognition and Machine IntelligenceAbstractNotating or transcribing a music piece is very important for musicians. It not only helps them to communicate among each other but also helps in understanding a piece. This is very much essential for improvisations and performances. This makes ...
Music Key Detection for Musical Audio
MMM '05: Proceedings of the 11th International Multimedia Modelling ConferenceThe key or the scale information of a piece of music provides important clues on its high level musical content, like harmonic and melodic context, which can be useful for music classification, retrieval or further content analysis. Researchers have ...
MIDI Conversion to Musical Notation
ICI '11: Proceedings of the 2011 First International Conference on Informatics and Computational IntelligenceNowadays, music has become an integral part of human life. People live with music surrounding them. Almost everywhere in the world there is music playing. Playing music is no longer the work of musician only, as all humans like music. Musical Instrument ...






Comments