Micah T. Taylor
ABSTRACT
We present an interactive algorithm and system (RESound) for sound propagation and rendering in virtual environments and media applications. RESound uses geometric propagation techniques for fast computation of propagation paths from a source to a listener and takes into account specular reflections, diffuse reflections, and edge diffraction. In order to perform fast path computation, we use a unified ray-based representation to efficiently trace discrete rays as well as volumetric ray-frusta. RESound further improves sound quality by using statistical reverberation estimation techniques. We also present an interactive audio rendering algorithm to generate spatialized audio signals. The overall approach can render sound in dynamic scenes allowing source, listener, and obstacle motion. Moreover, our algorithm is relatively easy to parallelize on multi-core systems. We demonstrate its performance on complex game-like and architectural environments.
- V. Algazi, R. Duda, and D. Thompson. The CIPIC HRTF Database. In IEEE ASSP Workshop on Applications of Signal Processing to Audio and Acoustics, 2001.Google Scholar
- J. B. Allen and D. A. Berkley. Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America, 65(4):943--950, April 1979.Google Scholar
- F. Antonacci, M. Foco, A. Sarti, and S. Tubaro. Fast modeling of acoustic reflections and diffraction in complex environments using visibility diagrams. In Proceedings of 12th European Signal Processing Conference (EUSIPCO'04), pages 1773--1776, September 2004.Google Scholar
- A. Chandak, L. Antani, M. Taylor, and D. Manocha. Fastv: From-point visibility culling on complex models. Eurographics Symposium on Rendering, 2009. Google ScholarDigital Library
- A. Chandak, C. Lauterbach, M. Taylor, Z. Ren, and D. Manocha. AD-Frustum: Adaptive Frustum Tracing for Interactive Sound Propagation. IEEE Transactions on Visualization and Computer Graphics, 14(6):1707--1722, Nov.-Dec. 2008. Google ScholarDigital Library
- R. Ciskowski and C. Brebbia. Boundary Element methods in acoustics. Computational Mechanics Publications and Elsevier Applied Science, 1991.Google Scholar
- P. R. Cook. Real Sound Synthesis for Interactive Applications. A. K. Peters, 2002. Google ScholarDigital Library
- B. Dalenback. Room acoustic prediction based on a unified treatment of diffuse and specular reflection. The Journal of the Acoustical Society of America, 100(2):899--909, 1996.Google ScholarCross Ref
- B.-I. Dalenb¨ack, M. Kleiner, and P. Svensson. A Macroscopic View of Diffuse Reflection. Journal of the Audio Engineering Society (JAES), 42(10):793--807, October 1994.Google Scholar
- N. Durlach and A. Mavor. Virtual Reality Scientific and Technological Challenges. National Academy Press, 1995.Google Scholar
- J. J. Embrechts. Broad spectrum diffusion model for room acoustics ray-tracing algorithms. The Journal of the Acoustical Society of America, 107(4):2068--2081, 2000.Google ScholarCross Ref
- C. F. Eyring. Reverberation time in "dead" rooms. The Journal of the Acoustical Society of America, 1(2A):217--241, January 1930.Google ScholarCross Ref
- T. Funkhouser, I. Carlbom, G. Elko, G. Pingali, M. Sondhi, and J. West. A beam tracing approach to acoustic modeling for interactive virtual environments. In Proc. of ACM SIGGRAPH, pages 21--32, 1998. Google ScholarDigital Library
- T. Funkhouser, N. Tsingos, and J.-M. Jot. Survey of Methods for Modeling Sound Propagation in Interactive Virtual Environment Systems. Presence and Teleoperation, 2003.Google Scholar
- M. Hodgson. Evidence of diffuse surface reflection in rooms. The Journal of the Acoustical Society of America, 88(S1):S185--S185, 1990.Google ScholarCross Ref
- B. Kapralos, M. Jenkin, and E. Milios. Acoustic Modeling Utilizing an Acoustic Version of Phonon Mapping. In Proc. of IEEE Workshop on HAVE, 2004.Google Scholar
- R. G. Kouyoumjian and P. H. Pathak. A uniform geometrical theory of diffraction for an edge in a perfectly conducting surface. Proc. of IEEE, 62:1448--1461, Nov. 1974.Google ScholarCross Ref
- A. Krokstad, S. Strom, and S. Sorsdal. Calculating the acoustical room response by the use of a ray tracing technique. Journal of Sound and Vibration, 8(1):118--125, July 1968.Google ScholarCross Ref
- K. Kunz and R. Luebbers. The Finite Difference Time Domain for Electromagnetics. CRC Press, 1993.Google Scholar
- H. Kuttruff. Acoustics. Routledge, 2007.Google Scholar
- S. Laine, S. Siltanen, T. Lokki, and L. Savioja. Accelerated beam tracing algorithm. Applied Acoustic, 70(1):172--181, 2009.Google ScholarCross Ref
- V. Larcher, O. Warusfel, J.-M. Jot, and J. Guyard. Study and comparison of efficient methods for 3-d audio spatialization based on linear decomposition of hrtf data. In Audio Engineering Society 108th Convention preprints, page preprint no. 5097, January 2000.Google Scholar
- C. Lauterbach, A. Chandak, and D. Manocha. Adaptive sampling for frustum-based sound propagation in complex and dynamic environments. In Proceedings of the 19th International Congress on Acoustics, 2007.Google Scholar
- C. Lauterbach, A. Chandak, and D. Manocha. Interactive sound rendering in complex and dynamic scenes using frustum tracing. IEEE Transactions on Visualization and Computer Graphics, 13(6):1672--1679, Nov.-Dec. 2007. Google ScholarDigital Library
- C. Lauterbach, S. Yoon, D. Tuft, and D. Manocha. RT-DEFORM: Interactive Ray Tracing of Dynamic Scenes using BVHs. IEEE Symposium on Interactive Ray Tracing, 2006.Google ScholarCross Ref
- J. Lehtinen. Time-domain numerical solution of the wave equation, 2003.Google Scholar
- R. B. Loftin. Multisensory perception: Beyond the visual in visualization. Computing in Science and Engineering, 05(4):56--58, 2003. Google ScholarDigital Library
- T. Moeck, N. Bonneel, N. Tsingos, G. Drettakis, I. Viaud-Delmon, and D. Alloza. Progressive perceptual audio rendering of complex scenes. In I3D '07: Proceedings of the 2007 symposium on Interactive 3D graphics and games, pages 189--196, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- P. Monk. Finite Element Methods for Maxwell's Equations. Oxford University Press, 2003.Google Scholar
- J. F. O'Brien, C. Shen, and C. M. Gatchalian. Synthesizing sounds from rigid-body simulations. In The ACM SIGGRAPH 2002 Symposium on Computer Animation, pages 175--181. ACM Press, July 2002. Google ScholarDigital Library
- N. Raghuvanshi, N. Galoppo, and M. C. Lin. Accelerated wave-based acoustics simulation. In ACM Solid and Physical Modeling Symposium, 2008. Google ScholarDigital Library
- N. Raghuvanshi and M. C. Lin. Interactive sound synthesis for large scale environments. In Symposium on Interactive 3D graphics and games, pages 101--108, 2006. Google ScholarDigital Library
- L. Savioja, J. Huopaniemi, T. Lokki, and R. Vaananen. Creating interactive virtual acoustic environments. Journal of the Audio Engineering Society (JAES), 47(9):675--705, September 1999.Google Scholar
- D. Schroder and T. Lentz. Real-Time Processing of Image Sources Using Binary Space Partitioning. Journal of the Audio Engineering Society (JAES), 54(7/8):604--619, July 2006.Google Scholar
- D. Schroder and A. Pohl. Real-time Hybrid Simulation Method Including Edge Diffraction. In EAA Auralization Symposium, Espoo, Finland, June 2009.Google Scholar
- K. Shoemake. Pl¨ucker coordinate tutorial. Ray Tracing News, 11(1), 1998.Google Scholar
- S. Siltanen, T. Lokki, S. Kiminki, and L. Savioja. The room acoustic rendering equation. The Journal of the Acoustical Society of America, 122(3):1624--1635, September 2007.Google ScholarCross Ref
- S. Siltanen, T. Lokki, and L. Savioja. Frequency domain acoustic radiance transfer for real-time auralization. Acta Acustica united with Acustica, 95:106--117(12), 2009.Google ScholarCross Ref
- J. E. Summers, R. R. Torres, and Y. Shimizu. Statistical-acoustics models of energy decay in systems of coupled rooms and their relation to geometrical acoustics. The Journal of the Acoustical Society of America, 116(2):958--969, August 2004.Google ScholarCross Ref
- P. Svensson and R. Kristiansen. Computational modelling and simulation of acoustic spaces. In 22nd International Conference: Virtual, Synthetic, and Entertainment Audio, June 2002.Google Scholar
- U. P. Svensson, R. I. Fred, and J. Vanderkooy. An analytic secondary source model of edge diffraction impulse responses . Acoustical Society of America Journal, 106:2331--2344, Nov. 1999.Google ScholarCross Ref
- M. Taylor, A. Chandak, Z. Ren, C. Lauterbach, and D. Manocha. Fast Edge-Diffraction for Sound Propagation in Complex Virtual Environments. In EAA Auralization Symposium, Espoo, Finland, June 2009.Google Scholar
- N. Tsingos. A versatile software architecture for virtual audio simulations. In International Conference on Auditory Display (ICAD), Espoo, Finland, 2001.Google Scholar
- N. Tsingos, T. Funkhouser, A. Ngan, and I. Carlbom. Modeling acoustics in virtual environments using the uniform theory of diffraction. In Proc. of ACM SIGGRAPH, pages 545--552, 2001. Google ScholarDigital Library
- N. Tsingos, E. Gallo, and G. Drettakis. Perceptual audio rendering of complex virtual environments. Technical Report RR-4734, INRIA, REVES/INRIA Sophia-Antipolis, Feb 2003.Google Scholar
- N. Tsingos, E. Gallo, and G. Drettakis. Perceptual audio rendering of complex virtual environments. ACM Trans. Graph., 23(3):249--258, 2004. Google ScholarDigital Library
- K. van den Doel. Sound Synthesis for Virtual Reality and Computer Games. PhD thesis, University of British Columbia, 1998.Google Scholar
- K. van den Doel, P. G. Kry, and D. K. Pai. Foleyautomatic: physically-based sound effects for interactive simulation and animation. In SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 537--544, New York, NY, USA, 2001. ACM Press. Google ScholarDigital Library
- M. Vorlander. Simulation of the transient and steady-state sound propagation in rooms using a new combined ray-tracing/image-source algorithm. The Journal of the Acoustical Society of America, 86(1):172--178, 1989.Google ScholarCross Ref
- I. Wald. Realtime Ray Tracing and Interactive Global Illumination. PhD thesis, Computer Graphics Group, Saarland University, 2004.Google Scholar
- M. Wand and W. Straßer. Multi-resolution sound rendering. In SPBG'04 Symposium on Point - Based Graphics 2004, pages 3--11, 2004. Google ScholarDigital Library
- E. Wenzel, J. Miller, and J. Abel. A software-based system for interactive spatial sound synthesis. In International Conference on Auditory Display (ICAD), Atlanta, GA, April 2000.Google Scholar
Index Terms
RESound: interactive sound rendering for dynamic virtual environments
Recommendations
Combining light animation with obscurances for glossy environments: Research Articles
Special Issue: The Very Best Papers from CASA 2004Obscurances is a powerful technique that approximates indirect global illumination in a much faster way than classic methods as radiosity or path tracing. In this method, the local environment of a point or a patch is sampled to estimate the amount of ...
Real-Time Volume-Based Ambient Occlusion
Real-time rendering can benefit from global illumination methods to make the 3D environments look more convincing and lifelike. On the other hand, the conventional global illumination algorithms for the estimation of the diffuse surface interreflection ...
Efficient image-based methods for rendering soft shadows
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniquesWe present two efficient imaged-based approaches for computation and display of high-quality soft shadows from area light sources. Our methods are related to shadow maps and provide the associated benefits. The computation time and memory requirements ...
Reviews
Pierre Jouvelot
Creating realistic virtual environments requires constant improvements to the synthesis of not only visual but also audio artifacts. RESound is an advanced interactive sound system that performs real-time simulation of sound propagation and rendering in configurations where the following parameters can be changed: acoustic space geometry, wall textures, and the sound emitters' and listeners' positions and speeds. The paper provides a synoptic description of the RESound software architecture. Sound propagation modeling takes into account the space where the sound source and the listener reside. RESound uses a geometric acoustics approach: propagation paths are computed using raytracing algorithms. Like light beams, sound paths generated at the source are modified by specular reflection on nondiffusive walls, diffraction around sharp objects, and diffusion by scattering on various materials. Since it is unrealistic to compute an exact propagation, this paper introduces reverberation approximations that use a statistical model for higher-order path alterations. RESound performs audio rendering by combining these sound paths to generate user audio signals, taking into account issues such as space changes, spatialization parameters, and the filtering introduced by the listener's own head. The paper ends with discussions of implementation details and how approximations used in RESound impact sound quality. This very interesting paper is easy to read. While it doesn't provide in-depth descriptions of the actual algorithms, it does thoroughly present a sophisticated physical audio synthesis pipeline that can be used for virtual reality environments and video games. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments