Precomputed wave simulation for real-time sound propagation of dynamic sources in complex scenes

Abstract

We present a method for real-time sound propagation that captures all wave effects, including diffraction and reverberation, for multiple moving sources and a moving listener in a complex, static 3D scene. It performs an offline numerical simulation over the scene and then applies a novel technique to extract and compactly encode the perceptually salient information in the resulting acoustic responses. Each response is automatically broken into two phases: early reflections (ER) and late reverberation (LR), via a threshold on the temporal density of arriving wavefronts. The LR is simulated and stored in the frequency domain, once per room in the scene. The ER accounts for more detailed spatial variation, by recording a set of peak delays/amplitudes in the time domain and a residual frequency response sampled in octave frequency bands, at each source/receiver point pair in a 5D grid. An efficient run-time uses this precomputed representation to perform binaural sound rendering based on frequency-domain convolution. Our system demonstrates realistic, wave-based acoustic effects in real time, including diffraction low-passing behind obstructions, sound focusing, hollow reverberation in empty rooms, sound diffusion in fully-furnished rooms, and realistic late reverberation.

References

1. Antonacci, F., Foco, M., Sarti, A., and Tubaro, S. 2004. Real time modeling of acoustic propagation in complex environments. Proceedings of 7th International Conference on Digital Audio Effects, 274--279.
2. Blauert, J. 1997. An introduction to binaural technology. In Binaural and Spatial Hearing in Real and Virtual Environments, R. Gilkey and T. R. Anderson, Eds. Lawrence Erlbaum, USA.
3. Botteldooren, D. 1995. Finite-difference time-domain simulation of low-frequency room acoustic problems. Acoustical Society of America Journal 98 (December), 3302--3308.
4. Calamia, P. 2009. Advances in Edge-Diffraction Modeling for Virtual-Acoustic Simulations. PhD thesis, Princeton University.
5. Chandak, A., Lauterbach, C., Taylor, M., Ren, Z., and Manocha, D. 2008. Ad-frustum: Adaptive frustum tracing for interactive sound propagation. IEEE Transactions on Visualization and Computer Graphics 14, 6, 1707--1722.
6. Emerit, M., Faure, J., Guerin, A., Nicol, R., Pallone, G., Philippe, P., and Virette, D. 2007. Efficient binaural filtering in QMF domain for BRIR. In AES 122th Convention.
7. Funkhouser, T., Tsingos, N., Carlbom, I., Elko, G., Sondhi, M., West, J. E., Pingali, G., Min, P., and Ngan, A. 2004. A beam tracing method for interactive architectural acoustics. The Journal of the Acoustical Society of America 115, 2, 739--756.
8. Gardner, W. G. 1998. Reverberation algorithms. In Applications of Digital Signal Processing to Audio and Acoustics, M. Kahrs and K. Brandenburg, Eds., 1 ed. Springer, 85--131.
9. Halmrast, T. 2007. Coloration due to reflections. further investigations. In International Congress on Acoustics.
10. Hartmann, W. M., and Wittenberg, A. 1996. On the externalization of sound images. The Journal of the Acoustical Society of America 99, 6 (June), 3678--3688.
11. James, D. L., Barbic, J., and Pai, D. K. 2006. Precomputed acoustic transfer: output-sensitive, accurate sound generation for geometrically complex vibration sources. ACM Transactions on Graphics 25, 3 (July), 987--995.
12. Jesteadt, W., Wier, C. C., and Green, D. M. 1977. Intensity discrimination as a function of frequency and sensation level. The Journal of the Acoustical Society of America 61, 1, 169--177.
13. Kinsler, L. E., Frey, A. R., Coppens, A. B., and Sanders, J. V. 2000. Fundamentals of acoustics, 4 ed. Wiley, December.
14. Kuttruff, H. 2000. Room Acoustics. Taylor & Francis, October.
15. Litovsky, R. Y., Colburn, S. H., Yost, W. A., and Guzman, S. J. 1999. The precedence effect. The Journal of the Acoustical Society of America 106, 4, 1633--1654.
16. Lokki, T. 2002. Physically-based Auralization. PhD thesis, Helsinki University of Technology.
17. Masterson, C., Kearney, G., and Boland, F. 2009. Acoustic impulse response interpolation for multichannel systems using dynamic time warping. In 35th AES Conference on Audio for Games.
18. Min, P., and Funkhouser, T. 2000. Priority-driven acoustic modeling for virtual environments. Computer Graphics Forum (September), 179--188.
19. Pope, J., Creasey, D., and Chalmers, A. 1999. Realtime room acoustics using ambisonics. In The Proceedings of the AES 16th International Conference on Spatial Sound Reproduction, Audio Engineering Society, 427--435.
20. Raghuvanshi, N., Narain, R., and Lin, M. C. 2009. Efficient and accurate sound propagation using adaptive rectangular decomposition. IEEE Transactions on Visualization and Computer Graphics 15, 5, 789--801.
21. Sakamoto, S., Ushiyama, A., and Nagatomo, H. 2006. Numerical analysis of sound propagation in rooms using the finite difference time domain method. The Journal of the Acoustical Society of America 120, 5, 3008.
22. Siltanen, S. 2005. Geometry Reduction in Room Acoustics Modeling. Master's thesis, Helsinki University of Technology.
23. Stavrakis, E., Tsingos, N., and Calamia, P. 2008. Topological sound propagation with reverberation graphs. Acta Acustica/Acustica - the Journal of the European Acoustics Association (EAA).
24. Svensson, U. P., Calamia, P., and Nakanishi, S. 2009. Frequency-domain edge diffraction for finite and infinite edges. In Acta Acustica/Acustica 95, 568--572.
25. Taylor, M. T., Chandak, A., Antani, L., and Manocha, D. 2009. Resound: interactive sound rendering for dynamic virtual environments. In MM '09: Proceedings of the seventeen ACM international conference on Multimedia, ACM, New York, NY, USA, 271--280.
26. Tsingos, N., Funkhouser, T., Ngan, A., and Carlbom, I. 2001. Modeling acoustics in virtual environments using the uniform theory of diffraction. In SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA, 545--552.
27. Tsingos, N., Dachsbacher, C., Lefebvre, S., and Dellepiane, M. 2007. Instant sound scattering. In Rendering Techniques (Proceedings of the Eurographics Symposium on Rendering).
28. Tsingos, N. 2009. Pre-computing geometry-based reverberation effects for games. In 35th AES Conference on Audio for Games.

Index Terms

• Computing methodologies

  • Modeling and simulation

    • Model development and analysis

    • Simulation types and techniques

      • Continuous simulation

• Hardware

  • Communication hardware, interfaces and storage

    • Signal processing systems