Tobias Lentz
Skip Abstract Section
Abstract
A real-time audio rendering system is introduced which combines a full room-specific simulation, dynamic crosstalk cancellation, and multitrack binaural synthesis for virtual acoustical imaging. The system is applicable for any room shape (normal, long, flat, coupled), independent of the a priori assumption of a diffuse sound field. This provides the possibility of simulating indoor or outdoor spatially distributed, freely movable sources and a moving listener in virtual environments. In addition to that, near-to-head sources can be simulated by using measured near-field HRTFs. The reproduction component consists of a headphone-free reproduction by dynamic crosstalk cancellation. The focus of the project is mainly on the integration and interaction of all involved subsystems. It is demonstrated that the system is capable of real-time room simulation and reproduction and, thus, can be used as a reliable platform for further research on VR applications.
- {1} D. R. Begault, "Challenges to the successful implementation of 3-D sound," Journal of the Audio Engineering Society, vol. 39, no. 11, pp. 864-870, 1991.Google Scholar
- {2} M. Naef, O. Staadt, and M. Gross, "Spatialized audio rendering for immersive virtual environments," in Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '02), pp. 65-72, Hong Kong, November 2002. Google ScholarDigital Library
- {3} C. Cruz-Neira, D. J. Sandin, T. A. DeFanti, R. V. Kenyon, and J. C. Hart, "The CAVE: audio visual experience automatic virtual environment," Communications of the ACM, vol. 35, no. 6, pp. 65-72, 1992. Google ScholarDigital Library
- {4} D. A. Burgess and J. C. Verlinden, "An architecture for spatial audio servers," in Proceedings of Virtual Reality Systems Conference (Fall '93), New York, NY, USA, November 1993.Google Scholar
- {5} J. D. Mulder and E. H. Dooijes, "Spatial audio in graphical applications," in Visualization in Scientific Computing, M. Göbel, H. Müller, and B. Urban, Eds., pp. 215-229, Springer, Wien, Austria, 1994.Google Scholar
- {6} Lake Huron, 2005, http://www.lake.com.au/.Google Scholar
- {7} L. Savioja, Modeling Techniques for Virtual Acoustics, Ph.D. thesis, Helsinki University of Technology, Helsinki, Finland, December 1999.Google Scholar
- {8} L. Savioja, J. Huopaniemi, T. Lokki, and R. Väänänen, "Creating interactive virtual acoustic environments," Journal of the Audio Engineering Society, vol. 47, no. 9, pp. 675-705, 1999.Google Scholar
- {9} T. Funkhouser, P. Min, and I. Carlbom, "Real-time acoustic modeling for distributed virtual environments," in Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '99), pp. 365-374, Los Angeles, Calif, USA, August 1999. Google ScholarDigital Library
- {10} R. L. Storms, "Npsnet-3D Sound Server: An Effective Use of the Auditory Channel," 1995.Google Scholar
- {11} H. Kuttruff, Room Acoustics, Elsevier Science Publisher, New York, NY, USA, 4th edition, 2000.Google Scholar
- {12} J. B. Allen and D. A. Berkley, "Image method for efficiently simulating small-room acoustics," The Journal of the Acoustical Society of America, vol. 65, no. 4, pp. 943-950, 1979.Google Scholar
- {13} J. Borish, "Extension of the image model to arbitrary polyhedra," The Journal of the Acoustical Society of America, vol. 75, no. 6, pp. 1827-1836, 1984.Google ScholarCross Ref
- {14} B.-I. L. Dalenbäck, "Room acoustic prediction based on a unified treatment of diffuse and specular reflection," The Journal of the Acoustical Society of America, vol. 100, no. 2, pp. 899-909, 1996.Google ScholarCross Ref
- {15} P.-A. Forsberg, "Fully discrete ray tracing," Applied Acoustics, vol. 18, no. 6, pp. 393-397, 1985.Google ScholarCross Ref
- {16} T. Funkhouser, N. Tsingos, I. Carlbom, et al., "A beam tracing method for interactive architectural acoustics," The Journal of the Acoustical Society of America, vol. 115, no. 2, pp. 739-756, 2004.Google ScholarCross Ref
- {17} G. M. Naylor, "ODEON--another hybrid room acoustical model," Applied Acoustics, vol. 38, no. 2-4, pp. 131-143, 1993.Google ScholarCross Ref
- {18} U. M. Stephenson, "Quantized pyramidal beam tracing--a new algorithm for room acoustics and noise immission prognosis," Acta Acustica United with Acustica, vol. 82, no. 3, pp. 517-525, 1996.Google Scholar
- {19} D. van Maercke, "Simulation of sound fields in time and frequency domain using a geometrical model," in Proceedings of the 12th International Congress on Acoustics (ICA '86), vol. 2, Toronto, Ontario, Canada, July 1986, paper E11-7.Google Scholar
- {20} M. Vorländer, "Simulation of the transient and steady state sound propagation in rooms using a new combined sound particle--image source algorithm," The Journal of the Acoustical Society of America, vol. 86, pp. 172-178, 1989.Google ScholarCross Ref
- {21} I. Bork, "A comparison of room simulation software--the 2nd round Robin on room acoustical computer simulation," Acta Acustica United with Acustica, vol. 86, no. 6, pp. 943-956, 2000.Google Scholar
- {22} M. Vorländer, "International round Robin on room acoustical computer simulations," in Proceedings of the 15th International Congress on Acoustics (ICA '95), pp. 689-692, Trondheim, Norway, June 1995.Google Scholar
- {23} H. Kuttruff, "A simple iteration scheme for the computation of decay constants in enclosures with diffusely reflecting boundaries," The Journal of the Acoustical Society of America, vol. 98, no. 1, pp. 288-293, 1995.Google ScholarCross Ref
- {24} C. L. Christensen and J. H. Rindel, "A new scattering method that combines roughness and diffraction effects," in Forum Acousticum, Budapest, Hungary, 2005.Google Scholar
- {25} R. Heinz, "Binaural room simulation based on an image source model with addition of statistical methods to include the diffuse sound scattering of walls and to predict the reverberant tail," Applied Acoustics, vol. 38, no. 2-4, pp. 145-159, 1993.Google ScholarCross Ref
- {26} Y. W. Lam, "A comparison of three reflection modelling methods used in room acoustics computer models," The Journal of the Acoustical Society of America, vol. 100, no. 4, pp. 2181-2192, 1996.Google ScholarCross Ref
- {27} M. Vorländer, "Ein Strahlverfolgungsverfahren zur Berechnung von Schallfeldern in Räumen," Acustica, vol. 65, no. 3, pp. 138-148, 1988.Google Scholar
- {28} J. S. Suh and P. A. Nelson, "Measurement of transient response of rooms and comparison with geometrical acoustic models," The Journal of the Acoustical Society of America, vol. 105, no. 4, pp. 2304-2317, 1999.Google ScholarCross Ref
- {29} U. P. Svensson, R. I. Fred, and J. Vanderkooy, "An analytic secondary source model of edge diffraction impulse responses," The Journal of the Acoustical Society of America, vol. 106, no. 5, pp. 2331-2344, 1999.Google ScholarCross Ref
- {30} N. Tsingos, T. Funkhouser, A. Ngan, and I. Carlbom, "Modeling acoustics in virtual environments using the uniform theory of diffraction," in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01), pp. 545-552, Los Angeles, Calif, USA, August 2001. Google ScholarDigital Library
- {31} U. M. Stephenson, Beugungssimulation ohne Rechenzeitexplosion: die Methode der quantisierten Pyramidenstrahlen; ein neues Berechnungsverfahren für Raumakustik und Lärmimmissionsprognose; Vergleiche, Ansätze, Lösungen, Ph.D. thesis, RWTH Aachen University, Aachen, Germany, 2004.Google Scholar
- {32} M. Slater, A. Steed, and Y. Chrysanthou, Computer Graphics and Virtual Environments: From Realism to Real-Time, Addison Wesley, New York, NY, USA, 2001. Google ScholarDigital Library
- {33} L. Cremer and H. A. Müller, Die wissenschaftlichen Grundlagen der Raumakustik--Band 1, S. Hirzel, Stuttgart, Germany, 2nd edition, 1978.Google Scholar
- {34} T. Akenine-Möller and E. Haines, Real-Time Rendering, A. K. Peters, Natick, Mass, USA, 2nd edition, 2002. Google ScholarDigital Library
- {35} J. D. Foley, A. van Dam, S. K. Feiner, and J. F. Hughes, Computer Graphics, Principles and Practice, Addison Wesley, Reading, Mass, USA, 2nd edition, 1996. Google ScholarDigital Library
- {36} R. Shumacker, R. Brand, M. Gilliland, and W. Sharp, "Study for applying computer-generated images to visual simulations," Report AFHRL-TR-69-14, U.S. Air Force Human Resources Laboratory, San Antonio, Tex, USA, 1969.Google ScholarCross Ref
- {37} T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, Cambridge, Mass, USA, 2nd edition, 2001. Google ScholarDigital Library
- {38} D. Schröder and T. Lentz, "Real-time processing of image sources using binary space partitioning," Journal of the Audio Engineering Society, vol. 54, no. 7-8, pp. 604-619, 2006.Google Scholar
- {39} R. Heinz, Entwicklung und Beurteilung von computergestützten Methoden zur binauralen Raumsimulation, Ph.D. thesis, RWTH Aachen University, Aachen, Germany, 1994.Google Scholar
- {40} J. S. Bradley and G. A. Soulodre, "The influence of late arriving energy on spatial impression," The Journal of the Acoustical Society of America, vol. 97, no. 4, pp. 2263-2271, 1995.Google ScholarCross Ref
- {41} J. H. Rindel, "Evaluation of room acoustic qualities and defects by use of auralization," in Proceedings of the 148th Meeting of the Acoustical Society of America, San Diego, Calif, USA, November 2004.Google Scholar
- {42} D. Schröder, P. Dross, and M. Vorländer, "A fast reverberation estimator for virtual environments," in Proceedings of the AES 30th International Conference, Saariselkä, Finland, March 2007.Google Scholar
- {43} T. Brookes and C. Treble, "The effect of non-symmetrical left/right recording pinnae on the perceived externalisation of binaural recordings," in Proceedings of the 118th Audio Engineering Society Convention, Barcelona, Spain, May 2005.Google Scholar
- {44} D. S. Brungart, W. M. Rabinowitz, and N. I. Durlach, "Auditory localization of a nearby point source," The Journal of the Acoustical Society of America, vol. 100, no. 4, p. 2593, 1996.Google ScholarCross Ref
- {45} A. Kulkarni and H. S. Colburn, "Role of spectral detail in sound-source localization," Nature, vol. 396, no. 6713, pp. 747-749, 1998.Google ScholarCross Ref
- {46} H. Lehnert and M. Richter, "Auditory virtual environment: simplified treatment of reflections," in Proceedings of the 15th International Congress on Acoustics (ICA '95), Trondheim, Norway, June 1995.Google Scholar
- {47} G. Romanenko and M. Vorländer, "Employment of spherical wave reflection coefficient in room acoustics," in IoA Symposium Surface Acoustics, Salford, UK, 2003.Google Scholar
- {48} C. Cruz-Neira, D. J. Sandin, and T. A. DeFanti, "Surround-screen projection-based virtual reality: the design and implementation of the CAVE," in Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '93), pp. 135-142, ACM Press, Anaheim, Calif, USA, August 1993. Google ScholarDigital Library
- {49} B. B. Bauer, "Stereophonic earphones and binaural loudspeakers," Journal of the Audio Engineering Society, vol. 9, no. 2, pp. 148-151, 1961.Google Scholar
- {50} O. Kirkeby, P. A. Nelson, and H. Hamada, "Local sound field reproduction using two closely spaced loudspeakers," The Journal of the Acoustical Society of America, vol. 104, no. 4, pp. 1973-1981, 1998.Google ScholarCross Ref
- {51} H. Møller, "Reproduction of artificial-head recordings through loudspeakers," Journal of the Audio Engineering Society , vol. 37, no. 1-2, pp. 30-33, 1989.Google Scholar
- {52} W. G. Gardner, 3-D audio using loudspeakers, Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, Mass, USA, 1997.Google Scholar
- {53} T. Lentz and O. Schmitz, "Realisation of an adaptive cross-talk cancellation system for a moving listener," in Proceedings of the 21st Audio Engineering Society Conference, St. Petersburg, Russia, June 2002.Google Scholar
- {54} T. Lentz and G. K. Behler, "Dynamic cross-talk cancellation for binaural synthesis in virtual reality environments," in Proceedings of the 117th Audio Engineering Society Convention, San Francisco, Calif, USA, October 2004.Google Scholar
- {55} Steinberg, "ASIO 2.0 Audio Streaming Input Output Development Kit," 2004.Google Scholar
- {56} T. Lentz, "Dynamic crosstalk cancellation for binaural synthesis in virtual reality environments," Journal of the Audio Engineering Society, vol. 54, no. 4, pp. 283-294, 2006.Google Scholar
- {57} T. Takeuchi, P. Nelson, O. Kirkeby, and H. Hamada, "The effects of reflections on the performance of virtual acoustic imaging systems," in Proceedings of the International Symposium on Active Control of Sound and Vibration (ACTIVE '97), pp. 955-966, Budapest, Hungary, August 1997.Google Scholar
- {58} D. B. Ward, "On the performance of acoustic crosstalk cancellation in a reverberant environment," The Journal of the Acoustical Society of America, vol. 110, no. 2, pp. 1195-1198, 2001.Google ScholarCross Ref
- {59} T. Lentz, J. Sokoll, and I. Assenmacher, "Performance of spatial audio using dynamic cross-talk cancellation," in Proceedings of the 119th Audio Engineering Society Convention, New York, NY, USA, October 2005.Google Scholar
- {60} W. G. Gardner, "Efficient convolution without input-output delay," Journal of the Audio Engineering Society, vol. 43, no. 3, pp. 127-136, 1995.Google Scholar
- {61} J. J. La Viola Jr., "A testbed for studying and choosing predictive tracking algorithms in virtual environments," in Proceedings of the 7th International Immersive Projection Technologies Workshop, 9th Eurographics Workshop on Virtual Environments , pp. 189-198, Zurich, Switzerland, May 2003. Google ScholarDigital Library
- {62} R. Azuma and G. Bishop, "A frequency-domain analysis of head-motion prediction," in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '95), pp. 401-408, ACM Press, Los Angeles, Calif, USA, August 1995. Google ScholarDigital Library
- {63} L. Chai, W. A. Hoff, and T. Vincent, "Three-dimensional motion and structure estimation using inertial sensors and computer vision for augmented reality," Presence: Teleoperators and Virtual Environments, vol. 11, no. 5, pp. 474-492, 2002. Google ScholarDigital Library
- {64} J.-R. Wu and M. Ouhyoung, "A 3D tracking experiment on latency and its compensation methods in virtual environments," in Proceedings of the 8th Annual ACM Symposium on User Interface and Software Technology (UIST '95), pp. 41-49, ACM Press, Pittsburgh, Pa, USA, November 1995. Google ScholarDigital Library
- {65} I. B. Witew, "Spatial variation of lateral measures in different concert halls," in Proceedings of the 18th International Congress on Acoustics (ICA '04), vol. 4, p. 2949, Kyoto, Japan, April 2004.Google Scholar
- {66} R. Azuma and G. Bishop, "Improving static and dynamic registration in an optical see-through HMD," in Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '94), pp. 197-204, ACM Press, New York, NY, USA, July 1994. Google ScholarDigital Library
- {67} W. Pompetzki, Psychoakustische Verifikation von Computer-modellen zur binauralen Raumsimulation, Ph.D. thesis, Ruhr-Universität Bochum, Bochum, Germany, 1993.Google Scholar
- {68} M. Vorländer and E. Mommertz, "Definition and measurement of random-incidence scattering coefficients," Applied Acoustics, vol. 60, no. 2, pp. 187-199, 2000.Google ScholarCross Ref
- {69} ISO 354, "Acoustics, Measurement of sound absorption in a reverberant room," 2003.Google Scholar
- {70} ISO/DIS 17497-1, "Acoustics Measurement of the sound scattering properties of surfaces--part 1: measurement of the randomincidence scattering coefficient in a reverberation room".Google Scholar
- {71} N. Tsingos, "Scalable perceptual mixing and filtering of audio signals using an augmented spectral representation," in Proceedings of the 8th International Conference on Digital Audio Effects (DAFx '05), Madrid, Spain, September 2005.Google Scholar
- {72} N. Tsingos, E. Gallo, and G. Drettakis, "Perceptual audio rendering of complex virtual environments," in Proceedings of the 31st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '04), pp. 249-258, Los Angeles, Calif, USA, August 2004. Google ScholarDigital Library
Index Terms
Virtual reality system with integrated sound field simulation and reproduction
Recommendations
Sound field reproduction around a scatterer in reverberation
ICASSP '09: Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal ProcessingWe devise a method for sound field reproduction (SFR) around a solid object in a reverberant room. Until now, work have focussed on reproducing sound in an empty listening space and, for the most part, in non-reverberant environments. However, in a ...
New method for sound field reproduction
A new audio system which reproduces reflections of actual concert halls is presented. The system is based on the symmetrical six-point sound-field analysis method and a sound-field simulation method through convolution. The system reproduces realistic ...
Locating virtual sound sources at arbitrary distances in real-time binaural reproduction
A real-time system for sound spatialization via headphones is presented. Conventional headphone spatialization techniques effectively place sources on the surface of a virtual sphere around the listener. In the new system, sources can be spatialized at ...
Comments