Interactive sound propagation using compact acoustic transfer operators

Abstract

We present an interactive sound propagation algorithm that can compute high orders of specular and diffuse reflections as well as edge diffractions in response to moving sound sources and a moving listener. Our formulation is based on a precomputed acoustic transfer operator, which we compactly represent using the Karhunen-Loeve transform. At runtime, we use a two-pass approach that combines acoustic radiance transfer with interactive ray tracing to compute early reflections as well as higher-order reflections and late reverberation. The overall approach allows accuracy to be traded off for improved performance at runtime, and has a low memory overhead. We demonstrate the performance of our algorithm on different scenarios, including an integration of our algorithm with Valve's Source game engine.

References

1. Alarcao, D., Santos, D., and Coelho, J. L. B. 2009. An auralization system for real time room acoustics simulation. In Proceedings of the Tecniacustica Conference.
2. Allen, J. B. and Berkley, D. A. 1979. Image method for efficiently simulating small-room acoustics. J. Acoust. Soc. Amer. 65, 4, 943--950.
3. Antani, L., Chandak, A., Taylor, M., and Manocha, D. 2011. Direct-to-Indirect acoustic radiance transfer. IEEE Trans. Vis. Comput. Graph. (To appear).
4. Bertram, M., Deines, E., Mohring, J., Jegorovs, J., and Hagen, H. 2005. Phonon tracing for auralization and visualization of sound. In Proceedings of the IEEE Visualization Conference. 151--158.
5. Bonneel, N., Drettakis, G., Tsingos, N., Viaud-Delmon, I., and James, D. L. 2008. Fast modal sounds with scalable frequency-domain synthesis. ACM Trans. Graph. 27, 3.
6. Botteldooren, D. 1995. Finite difference time domain simulation of low frequency room acoustic problems. J. Acoust. Soc. Amer. 98, 8, 3302--3308.
7. Chadwick, J., An, S., and James, D. L. 2009. Harmonic shells: A practical nonlinear sound model for near-rigid thin shells. ACM Trans. Graph. 28, 5.
8. Chandak, A., Antani, L., Taylor, M., and Manocha, D. 2009. Fastv: From-Point visibility culling on complex models. Comput. Graph. Forum 28, 1237--1246.
9. Ciskowski, R. and Brebbia, C. 1991. Boundary Element Methods in Acoustics. Computational Mechanics Publications and Elsevier Applied Science.
10. Foale, C. and Vamplew, P. 2007. Portal-Based sound propagation for first-person computer games. In Proceedings of the Australasian Conference on Interactive Entertainment. 9:1--9:8.
11. Funkhouser, T., Carlbom, I., Elko, G., Pingali, G., Sondhi, M., and West, J. 1998. A beam tracing approach to acoustic modeling for interactive virtual environments. In Proceedings of the SIGGRAPH Conference. 21--32.
12. Goral, C. M., Torrance, K. E., Greenberg, D. P., and Battaile, B. 1984. Modeling the interaction of light between diffuse surfaces. Comput. Graph. 18, 3, 213--222.
13. Hašan, M., Pellacini, F., and Bala, K. 2006. Direct-to-Indirect transfer for cinematic relighting. ACM Trans. Graph. 25, 3, 1089--1097.
14. Ihlenburg, F. 1998. Finite Element Analysis of Acoustic Scattering. Springer.
15. James, D. L., Barbič, J., and Pai, D. K. 2006. Precomputed acoustic transfer: Output-Sensitive, accurate sound generation for geometrically complex vibration sources. ACM Trans. Graph. 25, 3, 987--995.
16. Kajiya, J. T. 1986. The rendering equation. Comput. Graph. 20, 4, 143--150.
17. Kapralos, B., Jenkin, M., and Milios, E. 2004. Sonel mapping: Acoustic modeling utilizing an acoustic version of photon mapping. In Proceedings of the IEEE International Workshop on Haptics Audio Visual Environments and their Applications.
18. Kristensen, A. W., Akenine-Möller, T., and Jensen, H. W. 2005. Precomputed local radiance transfer for real-time lighting design. ACM Trans. Graph. 24, 3, 1208--1215.
19. Kuttruff, H. 1991. Room Acoustics. Elsevier Science Publishing Ltd.
20. Kuttruff, H. 1995. A simple iteration scheme for the computation of decay constants in enclosures with diffusely reflecting boundaries. J. Acoust. Soc. Amer. 98, 1, 288--293.
21. Kuttruff, H. K. 1993. Auralization of impulse responses modeled on the basis of ray-tracing results. J. Audio Engin. Soc. 41, 11, 876--880.
22. Laine, S., Siltanen, S., Lokki, T., and Savioja, L. 2009. Accelerated beam tracing algorithm. Appl. Acoust. 70, 1, 172--181.
23. Lehtinen, J., Zwicker, M., Turquin, E., Kontkanen, J., Durand, F., Sillion, F. X., and Aila, T. 2008. A meshless hierarchical representation for light transport. ACM Trans. Graph. 27, 3, 1--9.
24. Lentz, T., Schroeder, D., Vorlander, M., and Assenmacher, I. 2007. Virtual reality system with integrated sound field simulation and reproduction. EURASIP J. Appl. Signal Process. 2007, 1.
25. Loève, M. 1978. Probability Theory Vol. II. Springer.
26. Moeck, T., Bonneel, N., Tsingos, N., Drettakis, G., Viaud-Delmon, I., and Aloza, D. 2007. Progressive perceptual audio rendering of complex scenes. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games.
27. Nosal, E.-M., Hodgson, M., and Ashdown, I. 2004. Improved algorithms and methods for room sound-field prediction by acoustical radiosity in arbitrary polyhedral rooms. J. Acoust. Soc. Amer. 116, 2, 970--980.
28. Raghuvanshi, N. and Lin, M. C. 2006. Interactive sound synthesis for large scale environments. In Proceedings of the Symposium on Interactive 3D Graphics and Games.
29. Raghuvanshi, N., Narain, R., and Lin, M. C. 2009. Efficient and accurate sound propagation using adaptive rectangular decomposition. IEEE Trans. Vis. Comput. Graph. 15, 789--801.
30. Raghuvanshi, N., Snyder, J., Mehra, R., Lin, M., and Govindaraju, N. 2010. Precomputed wave simulation for real-time sound propagation of dynamic sources in complex scenes. ACM Trans. Graph. 29, 4, 68:1--68:11.
31. Savioja, L., Huopaniemi, J., Lokki, T., and Väänänen, R. 1999. Creating interactive virtual acoustic environments. J. Audio Engin. Soc. 47, 9, 675--705.
32. Savioja, L., Rinne, T., and Takala, T. 1994. Simulation of room acoustics with a 3-D finite difference mesh. In Proceedings of the International Computer Music Conference. 463--466.
33. Siltanen, S., Lokki, T., Kiminki, S., and Savioja, L. 2007. The room acoustic rendering equation. J. Acoust. Soc. Amer. 122, 3, 1624--1635.
34. Siltanen, S., Lokki, T., and Savioja, L. 2009. Frequency domain acoustic radiance transfer for real-time auralization. In Proceedings of Acta Acustica united with Acustica 95, 106--117.
35. Sloan, P.-P., Kautz, J., and Snyder, J. 2002. Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments. ACM Trans. Graph. 21, 3, 527--536.
36. Stavrakis, E., Tsingos, N., and Calamia, P. 2008. Topological sound propagation with reverberation graphs. In Proceedings of Acta Acustica united with Acustica.
37. Stephenson, U. M. 2010. An analytically derived sound particle diffraction model. In Proceedings of Acta Acustica united with Acustica 96, 1051--1068.
38. Stephenson, U. M. and Svensson, U. P. 2007. An improved energetic approach to diffraction based on the unvertainty principle. In Proceedings of the 19th International Congress on Acoustics (ICA).
39. Summers, J. E., Torres, R. R., and Shimizu, Y. 2004. Statistical-acoustics models of energy decay in systems of coupled rooms and their relation to geometrical acoustics. J. Acoust. Soc. Amer. 116, 2, 958--969.
40. Svensson, U. P., Fred, R. I., and Vanderkooy, J. 1999. An analytic secondary source model of edge diffraction impulse responses. J. Acoust. Soc. Amer. 106, 2331--2344.
41. Taylor, M. T., Chandak, A., Antani, L., and Manocha, D. 2009. Resound: interactive sound rendering for dynamic virtual environments. In Proceedings of the ACM Multimedia Conference. 271--280.
42. Tsingos, N. 2007. Perceptually-Based auralization. In Proceedings of the International Congress on Acoustics.
43. Tsingos, N. 2009. Precomputing geometry-based reverberation effects for games. In Proceedings of the Audio Engineering Society Conference: Audio for Games.
44. Tsingos, N., Funkhouser, T., Ngan, A., and Carlbom, I. 2001. Modeling acoustics in virtual environments using the uniform theory of diffraction. In Proceedings of the SIGGRAPH Conference. 545--552.
45. Tsingos, N., Gallo, E., and Drettakis, G. 2004. Perceptual audio rendering of complex virtual environments. ACM Trans. Graph. 23, 3.
46. Tsingos, N. and Gascuel, J.-D. 1997. A general model for the simulation of room acoustics based on hierachical radiosity. In Proceedings of the ACM SIGGRAPH 97.
47. Wallace, J. R., Cohen, M. F., and Greenberg, D. P. 1987. A two-pass solution to the rendering equation: A synthesis of ray tracing and radiosity methods. Comput. Graph. 21, 4, 311--320.
48. Wang, Ye; Vilermo, M. 2003. Modified discrete cosine transform: Its implications for audio coding and error concealment. J. Audio Engin. Soc 51, 1/2, 52--61.

Index Terms

• Applied computing

  • Arts and humanities

    • Sound and music computing

  • Document management and text processing

    • Document capture

      • Graphics recognition and interpretation

• Computing methodologies

  • Computer graphics

    • Rendering

      • Reflectance modeling

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval

        • Music retrieval