Shiguang Liu
Abstract
A typical rainfall scenario contains tens of thousands of dynamic sound sources. A characteristic of the large-scale scene is the strong randomness in raindrop distribution, which makes it notoriously expensive to synthesize such sounds with purely physical methods. Moreover, the raindrops hitting different surfaces (liquid or various solids) can emit distinct sounds, for which prior methods with unified impact sound models are ill-suited.
In this paper, we present a physically-based statistical simulation method to synthesize realistic rain sound, which respects surface materials. We first model the raindrop sound with two mechanisms, namely the initial impact and the subsequent pulsation of entrained bubbles. Then we generate material sound textures (MSTs) based on a specially designed signal decomposition and reconstruction model. This allows us to distinguish liquid surface with bubble sound and different solid surfaces with MSTs. Furthermore, we build a basic rain sound (BR-sound) bank with the proposed raindrop sound clustering method based on a statistical model, and design a sound source activator for simulating spatial propagation in an efficient manner. This novel method drastically decreases the computational cost while producing convincing sound results. Various experiments demonstrate the effectiveness of our sound simulation model.
Supplemental Material
- C. Cao, Z. Ren, C. Schissler, D. Manocha, and K. Zhou. 2016. Interactive sound propagation with bidirectional path tracing. ACM Transactions on Graphics 35, 6 (2016), 180:1--180:11. Google Scholar
Digital Library
- J.N. Chadwick and D.L. James. 2011. Animating fire with sound. ACM Transactions on Graphics 30, 4 (2011), 84:1--84:8. Google ScholarDigital Library
- H. Cheng and S. Liu. 2019. Haptic force guided sound synthesis in multisensory virtual reality (VR) simulation for rigid-fluid interaction. In Proceedings of IEEE VR. 1--9.Google Scholar
- P.R. Cook. 1997. Physically informed sonic modeling (PHISM): Synthesis of percussive sounds. Computer Music Journal 21, 3 (1997), 38--49.Google ScholarCross Ref
- P.R. Cook. 2002. Modeling bill's gait: Analysis and parametric synthesis of walking sounds. In Proceedings of Audio Engineering Society Conference. 73--78.Google Scholar
- N. Dingle and Y. Lee. 1972. Terminal fallspeeds of raindrops. Journal of Applied Meteorology 11, 5 (1972), 877--879.Google ScholarCross Ref
- K.V.D. Doel. 2005. Physically-based models for liquid sounds. ACM Transactions on Applied Perception 2, 4 (2005), 534--546. Google ScholarDigital Library
- K. Dragomiretskiy and D. Zosso. 2014. Variational mode decomposition. IEEE Transactions on Signal Processing 62, 3 (2014), 531--544. Google ScholarDigital Library
- A. Farnell. 2010. Designing Sound. The MIT Press. Google ScholarDigital Library
- G.J. Franz. 1959. Splashes as sources of sound in liquids. Journal of the Acoustical Society of America 31, 8 (1959), 1080--1096.Google ScholarCross Ref
- Y.P. Guo and J.F. Williams. 1991. A theoretical study on drop impact sound and rain noise. Journal of Fluid Mechanics 227 (1991), 345--355.Google ScholarCross Ref
- M. S. Howe and N. A. A. Hagen. 2011. On the impact noise of a drop falling on water. Journal of Sound and Vibration 330, 4 (2011), 625--635.Google ScholarCross Ref
- N.E. Huang, Z. Shen, S.R. Long, M.C. Wu, H.H. Shi, Q. Zheng, N.C. Yen, C.T. Chi, and H.H. Liu. 1998. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. 454, 1971 (1998), 903--995.Google Scholar
- P.W. Jacobus. 1991. Underwater sound radiation from large raindrops. Technical Report. Naval Postgraduate School Monterey CA.Google Scholar
- T.R. Langlois, C. Zheng, and D.L. James. 2016. Toward animating water with complex acoustic bubbles. ACM Transactions on Graphics 35, 4 (2016), 95:1--95:13. Google ScholarDigital Library
- T. Leighton. 1994. The acoustic bubble. Academic press.Google Scholar
- T. Lentz, D. Schröder, M. Vorländer, and I. Assenmacher. 2007. Virtual reality system with integrated sound field simulation and reproduction. EURASIP Journal on Advances in Signal Processing 2007, 1 (2007), 1--19. Google ScholarDigital Library
- S. Liu and Z. Yu. 2015. Sounding fire for immersive virtual reality. Virtual Reality 19, 3--4 (2015), 291--302. Google ScholarDigital Library
- M.S. Longuet-Higgins. 1990. An analytic model of sound production by raindrops. Fluid Mechanics 214 (1990), 395--410.Google ScholarCross Ref
- J.S. Marshall and W.M.K. Palmer. 1948. The distribution of raindrops with size. Journal of Meteorology 5 (1948), 165--166.Google ScholarCross Ref
- H. Medwin, J.A. Nystuen, P.W. Jacobus, L.H. Ostwald, and D.E. Snyder. 1992. The anatomy of underwater rain noise. Journal of the Acoustical Society of America 92, 3 (1992), 1613--1623.Google ScholarCross Ref
- R. Mehra, N. Raghuvanshi, L. Antani, A. Chandak, S. Curtis, and D. Manocha. 2013. Wave-based sound propagation in large open scenes using an equivalent source formulation. ACM Transactions on Graphics 32, 2 (2013), 19:1--19:13. Google ScholarDigital Library
- M. Minnaert. 1933. On musical air-bubbles and the sounds of running water. Philosophical Magazine 16, 104 (1933), 235--248.Google Scholar
- A. Misra, P. R. Cook, and G. Wang. 2006. TAPESTREA: sound scene modeling by example. In Proceedings of SIGGRAPH Sketches. 177. Google ScholarDigital Library
- W. Moss, H. Yeh, J.M. Hong, M.C. Lin, and D. Manocha. 2010. Sounding liquids: Automatic sound synthesis from fluid simulation. ACM Transactions on Graphics 29, 3 (2010), 21:1--21:13. Google ScholarDigital Library
- J.A. Nystuen. 1986. Rainfall measurements using underwater ambient noise. Journal of the Acoustical Society of America 79, 4 (1986), 972--982.Google ScholarCross Ref
- J.A. Nystuen. 1991. Ambient sound in the ocean produced by heavy precipitation and the subsequent predictability of rainfall rate. Journal of the Acoustical Society of America 90, 4 (1991), 2301.Google ScholarCross Ref
- L. Peltola, C. Erkut, P.R. Cook, and V. Valimaki. 2007. Synthesis of hand clapping sounds. IEEE Transactions on Audio, Speech, and Language Processing 15, 3 (2007), 1021--1029. Google ScholarDigital Library
- A. Prosperetti, L.A. Crum, and H.C. Pumphrey. 1989. The underwater noise of rain. Journal of Geophysical Research Oceans 94, C3 (1989), 3255--3259.Google ScholarCross Ref
- H.C. Pumphrey, L.A. Crum, and L. Bjørnø. 1989. Underwater sound produced by individual drop impacts and rainfall. Journal of the Acoustical Society of America 85, 4 (1989), 1518--1526.Google ScholarCross Ref
- H.C. Pumphrey and P.A. Elmore. 1990. The entrainment of bubbles by drop impacts. Journal of Fluid Mechanics 220 (1990), 539--567.Google ScholarCross Ref
- N. Raghuvanshi and J. Snyder. 2014. Parametric wave field coding for precomputed sound propagation. ACM Transactions on Graphics 33, 4 (2014), 38:1--38:11. Google ScholarDigital Library
- Z. Ren, H. Yeh, and M.C. Lin. 2013. Example-guided physically based modal sound synthesis. ACM Transactions on Graphics 32, 1 (2013), 1:1--1:16. Google ScholarDigital Library
- C. Roads. 1988. Introduction to granular synthesis. Computer Music Journal 12, 2 (1988), 11--13.Google Scholar
- C. Schissler, R. Mehra, and D. Manocha. 2014. High-order diffraction and diffuse reflections for interactive sound propagation in large environments. ACM Transactions on Graphics 33, 4 (2014), 39:1--39:12. Google ScholarDigital Library
- D. Schwarz and N. Schnell. 2010. Descriptor-based sound texture sampling. In Proceedings of International Conference on Sound and Music Computing. 510--515.Google Scholar
- C.D. Scofield. 1992. Oscillating microbubbles created by water drops falling on fresh and salt water: amplitude, damping and the effects of temperature and salinity. Ph.D. Dissertation. Naval Postgraduate School.Google Scholar
- G. Strobl, G. Eckel, and D. Rocchesso. 2006. Sound texture modeling: A survey. In Proceedings of International Conference on Sound and Music Computing. 1--7.Google Scholar
- L.L. Thompson. 2006. A review of finite-element methods for time-harmonic acoustics. Journal of the Acoustical Society of America 20, 3 (2006), 1315--1330.Google ScholarCross Ref
- C. Verron and G. Drettakis. 2012. Procedural audio modeling for particle-based environmental effects. In Proceedings of Audio Engineering Society Convention 133.Google Scholar
- Q. Yin and S. Liu. 2018. Sounding solid combustibles: non-premixed flame sound synthesis for different solid combustibles. IEEE Transactions on Visualization and Computer Graphics 24, 2 (2018), 1179--1189.Google ScholarCross Ref
- Z. Zhang, N. Raghuvanshi, J. Snyder, and S. Marschner. 2018. Ambient sound propagation. ACM Transactions on Graphics 37, 6 (2018), 184:1--184:10. Google ScholarDigital Library
- C. Zheng and D.L. James. 2009. Harmonic fluids. ACM Transactions on Graphics 28, 3 (2009), 37:1--37:12. Google ScholarDigital Library
- A. Zita. 2003. Computational real-time sound synthesis of rain. Master's thesis. Linköping University.Google Scholar
Index Terms
Physically-based statistical simulation of rain sound
Recommendations
Real-time rendering of decorative sound textures for soundscapes
Audio recordings contain rich information about sound sources and their properties such as the location, loudness, and frequency of events. One prevalent component in sound recordings is the sound texture, which contains a massive number of events. In ...
Perceptual Evaluation of Synthesized Sound Effects
Sound synthesis is the process of generating artificial sounds through some form of simulation or modelling. This article aims to identify which sound synthesis methods achieve the goal of producing a believable audio sample that may replace a recorded ...
Real-time rendering of aerodynamic sound using sound textures based on computational fluid dynamics
In computer graphics, most research focuses on creating images. However, there has been much recent work on the automatic generation of sound linked to objects in motion and the relative positions of receivers and sound sources. This paper proposes a ...
Comments