Sriram Sami
Yimin Dai
ABSTRACT
Eavesdropping on private conversations is one of the most common yet detrimental threats to privacy. A number of recent works have explored side-channels on smart devices for recording sounds without permission. This paper presents LidarPhone, a novel acoustic side-channel attack through the lidar sensors equipped in popular commodity robot vacuum cleaners. The core idea is to repurpose the lidar to a laser-based microphone that can sense sounds from subtle vibrations induced on nearby objects. LidarPhone carefully processes and extracts traces of sound signals from inherently noisy laser reflections to capture privacy sensitive information (such as speech emitted by a victim's computer speaker as the victim is engaged in a teleconferencing meeting; or known music clips from television shows emitted by a victim's TV set, potentially leaking the victim's political orientation or viewing preferences). We implement LidarPhone on a Xiaomi Roborock vacuum cleaning robot and evaluate the feasibility of the attack through comprehensive real-world experiments. We use the prototype to collect both spoken digits and music played by a computer speaker and a TV soundbar, of more than 30k utterances totaling over 19 hours of recorded audio. LidarPhone achieves approximately 91% and 90% average accuracies of digit and music classifications, respectively.
- Martín Abadi et al. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org.Google Scholar
- S. Abhishek Anand, Chen Wang, Jian Liu, Nitesh Saxena, and Yingying Chen. 2019. Spearphone: A Speech Privacy Exploit via Accelerometer-Sensed Reverberations from Smartphone Loudspeakers. arXiv:1907.05972 [cs] (July 2019). http://arxiv.org/abs/1907.0597200003 arXiv: 1907.05972.Google Scholar
- David P Baron. 2006. Persistent media bias. Journal of Public Economics 90, 1--2 (2006), 1--36.Google ScholarCross Ref
- Sören Beye. 2020. Hypfer/Valetudo. https://github.com/Hypfer/ValetudoGoogle Scholar
- Steven Boll. 1979. Suppression of acoustic noise in speech using spectral subtraction. IEEE Transactions on acoustics, speech, and signal processing 27, 2 (1979), 113--120.Google ScholarCross Ref
- Bosch. 2019. ISC-BPR2 - Blue Line Gen2 PIR Motion Detectors. http://resource.boschsecurity.com/documents/BlueLine_Gen_2_Data_sheet_enUS_2603228171.pdf.Google Scholar
- P Castellini, M Martarelli, and EP Tomasini. 2006. Laser Doppler Vibrometry: Development of advanced solutions answering to technology's needs. Mechanical systems and signal processing 20, 6 (2006), 1265--1285.Google Scholar
- François Chollet et al. 2015. Keras. https://keras.io.Google Scholar
- Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham Mysore, Fredo Durand, and William T. Freeman. 2014. The Visual Microphone: Passive Recovery of Sound from Video. ACM Transactions on Graphics (Proc. SIGGRAPH) 33, 4 (2014), 79:1--79:10.Google Scholar
- Dragonmouth and Laura Tucker. 2018. Be Careful of Your Robot Vacuum - It Could Be Spying on You. https://www.maketecheasier.com/robot-vacuum-spying/Google Scholar
- ECOVACS. 2020. DEEBOT OZMO 960 - ECOVACS. https://www.ecovacs.com/us/deebot-robotic-vacuum-cleaner/DEEBOT-OZMO-960Google Scholar
- ECOVACS. 2020. DEEBOT OZMO T5 - ECOVACS. https://www.ecovacs.com/us/deebot-robotic-vacuum-cleaner/DEEBOT-OZMO-T5Google Scholar
- ECOVACS. 2020. DEEBOT OZMO T8 AIVI - ECOVACS. https://www.ecovacs.com/us/deebot-robotic-vacuum-cleaner/DEEBOT-OZMO-T8-AIVIGoogle Scholar
- Paul Eisenstein. 2013. Seven Best Cars for Front Crash Avoidance. http://www.thedetroitbureau.com/2013/09/seven-best-cars-for-front-crash-avoidance/.Google Scholar
- Dan Ellis. 2003. Clean Digits. https://www.ee.columbia.edu/~dpwe/sounds/tidigits/Google Scholar
- Eduardo Fonseca, Rong Gong, Dmitry Bogdanov, Olga Slizovskaia, Emilia Gómez Gutiérrez, and Xavier Serra. 2017. Acoustic scene classification by ensembling gradient boosting machine and convolutional neural networks. In Virtanen T, Mesaros A, Heittola T, Diment A, Vincent E, Benetos E, Martinez B, editors. Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE2017); 2017 Nov 16; Munich, Germany. Tampere (Finland): Tampere University of Technology; 2017. p. 37--41. Tampere University of Technology.Google Scholar
- Market Research Future. 2020. Home theatre market research report-forecast 2023 | home theatre industry. https://www.marketresearchfuture.com/reports/home-theatre-market-4121Google Scholar
- Dave Gans. 2017. 3 Tips on Where to Place a Subwoofer. https://www.klipsch.com/blog/place-a-subwoofer-3-tipsGoogle Scholar
- Dennis Giese. 2018. Having fun with IoT: Reverse Engineering and Hacking of Xiaomi IoT Devices. https://dontvacuum.me/talks/DEFCON26/DEFCON26-Having_fun_with_IoT-Xiaomi.pdfGoogle Scholar
- D Giese. 2018. Having fun with IoT: reverse engineering and hacking of xiaomi IoT devices.Google Scholar
- Jun Han, Albert Jin Chung, and Patrick Tague. 2017. Pitchln: eavesdropping via intelligible speech reconstruction using non-acoustic sensor fusion. In Proceedings of the 16th ACM/IEEE International Conference on Information Processing in Sensor Networks - IPSN '17. ACM Press, Pittsburgh, Pennsylvania, 181--192. Google ScholarDigital Library
- Taylor Hatmaker. 2018. A vacuum vulnerability could mean your Roomba knockoff is hoovering up surveillance. https://techcrunch.com/2018/07/19/vacuum-vulnerability-hack-diqee-positive-technologies/Google Scholar
- Chris Heinonen. 2015. Your Privacy, Your Devices, and You. http://thewirecutter.com/blog/your-privacy-your-devices-and-you/.Google Scholar
- Lars Hertel, Huy Phan, and Alfred Mertins. 2016. Comparing Time and Frequency Domain for Audio Event Recognition Using Deep Learning. (2016).Google Scholar
- Zohar Jackson, César Souza, Jason Flaks, Yuxin Pan, Hereman Nicolas, and Adhish Thite. 2018. Jakobovski/free-spoken-digit-dataset: v1.0.8. Google ScholarCross Ref
- Kaspersky. 2018. Xiaomi Mi Robot vacuum cleaner hacked. https://www.kaspersky.com/blog/xiaomi-mi-robot-hacked/20632/Google Scholar
- Qiuqiang Kong, Yin Cao, Turab Iqbal, Yong Xu, Wenwu Wang, and Mark D Plumbley. 2019. Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems. arXiv preprint arXiv:1904.03476 (2019).Google Scholar
- Qiuqiang Kong, Turab Iqbal, Yong Xu, Wenwu Wang, and Mark D Plumbley. 2018. DCASE 2018 challenge surrey cross-task convolutional neural network baseline. arXiv preprint arXiv:1808.00773 (2018).Google Scholar
- Kurt Konolige, Joseph Augenbraun, Nick Donaldson, Charles Fiebig, and Pankaj Shah. 2008. A low-cost laser distance sensor. In 2008 IEEE international conference on robotics and automation. IEEE, 3002--3008.Google ScholarCross Ref
- Kurt Konolige, Joseph Augenbraun, Nick Donaldson, Charles Fiebig, and Pankaj Shah. 2008. A low-cost laser distance sensor. In 2008 IEEE International Conference on Robotics and Automation. IEEE, Pasadena, CA, USA, 3002--3008. Google ScholarCross Ref
- M Korkmaz, A Durdu, and YE Tusun. 2018. Sensor Comparison for a Real-Time SLAM Application. International Journal of Information and Electronics Engineering 8, 1 (2018).Google ScholarCross Ref
- Steven Kreuzer. 2019. PTPd. http://ptpd.sourceforge.net/.Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (Lake Tahoe, Nevada) (NIPS'12). Curran Associates Inc., Red Hook, NY, USA, 1097--1105.Google ScholarDigital Library
- Lenovo. 2020. Lenovo X1: The Newest Lidar-Based Robot Vacuum with Strong Suction and Smart Features. https://smartrobotreviews.com/reviews/lenovo-x1-robot-vacuum-features-review.htmlGoogle Scholar
- LG. 2020. LG SL5Y : SL5Y 2.1 Channel 400W Sound Bar w/ DTS Virtual: X & High Resolution Audio. https://www.lg.com/us/home-audio/lg-SL5YGoogle Scholar
- Logitech. 2020. Logitech Z623 2.1 Speaker System with Subwoofer. https://www.logitech.com/en-us/product/speaker-system-z623Google Scholar
- Yongsen Ma, Gang Zhou, and Shuangquan Wang. 2019. WiFi sensing with channel state information: A survey. ACM Computing Surveys (CSUR) 52, 3 (2019), 1--36.Google ScholarDigital Library
- Meghan McDonough. 2018. Roborock s5 robot vacuum review: jack-of-all-trades, master of none. https://www.tomsguide.com/us/roborock-s5-robot-vacuum,review-6274.htmlGoogle Scholar
- William McGrath. 2005. Technique and device for through-the-wall audio surveillance. US Patent App. 11/095,122.Google Scholar
- Yan Michalevsky, Dan Boneh, and Gabi Nakibly. 2014. Gyrophone: Recognizing Speech from Gyroscope Signals. In 23rd USENIX Security Symposium (USENIX Security 14). USENIX Association, San Diego, CA, 1053--1067.Google ScholarDigital Library
- Ralph P Muscatell. 1984. Laser microphone. US Patent 4,479,265.Google Scholar
- Television Music. 2016. NPR Morning Edition Theme Song. https://www.youtube.com/watch?v=otGmpQpiVjw&list=PLtxqRD7TEIMdPsuCW1fK30zQzdJ9gBtn&index=10&t=0sGoogle Scholar
- Television Music. 2018. The Situation Room Theme Music - CNN. https://www.youtube.com/watch?v=IpoXtx6mkPY&list=PLtxqRD7TEIMdPsuCVV1fK30zQzdJ9gBtn&index=2&t=0sGoogle Scholar
- Neato Robotics. 2020. Botvac™ Connected | Neato Robotics | Singapore |. http://www.neatorobotics.com.sg/botvac-connected/Google Scholar
- Neato Robotics. 2020. Neato D4 robot vacuum - Neato - Intelligent Robot Vacuums. https://neatorobotics.com/products/neato-d4/Google Scholar
- Neato Robotics. 2020. Neato D6 robot vacuum - Neato - Intelligent Robot Vacuums. https://neatorobotics.com/products/neato-d6/Google Scholar
- O'Donnell. 2019. IoT Robot Vacuum Vulnerabilities Let Hackers Spy on Victims. https://threatpost.com/iot-robot-vacuum-vulnerabilities-let-hackers-spy-on-victims/134179/Google Scholar
- University of Michigan. 2020. Where do news sources fall on the political bias spectrum? https://guides.lib.umich.edu/c.php?g=637508&p=4462444Google Scholar
- Theodor Olsson and Albin Larsson Forsberg. 2019. IoT Offensive Security Penetration Testing. (2019).Google Scholar
- Sangwook Park, Seongkyu Mun, Younglo Lee, and Hanseok Ko. 2017. Acoustic scene classification based on convolutional neural network using double image features. In Proc. of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE2017). 98--102.Google Scholar
- Pery Pearson. 1993. Sound Sampling. http://www.hitl.washington.edu/projects/knowledge_base/virtual-worlds/EVE/I.B.3.a.SoundSampling.html.Google Scholar
- Avian Research. 2007. Netcat: the TCP/IP swiss army. https://nc110.sourceforge.io/Google Scholar
- Neato Robotics. 2020. Neato D7 robot vacuum - Neato - Intelligent Robot Vacuums. https://neatorobotics.com/products/neato-d7/Google Scholar
- Nirupam Roy and Romit Roy Choudhury. 2016. Listening through a Vibration Motor. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services - MobiSys '16. ACM Press, Singapore, Singapore, 57--69. Google ScholarDigital Library
- RpLidar. 2014. Low cost 360 degree 2D laser scanner (Lidar) system-Introduction and Datasheet. Robopeak Team (2014).Google Scholar
- Carl Runge. 1901. Über empirische Funktionen und die Interpolation zwischen äquidistanten Ordinaten. Zeitschrift für Mathematik und Physik 46, 224--243 (1901), 20.Google Scholar
- Argo-A Security. 2020. Long-Range Laser Listening Device. http://argoasecurity.com/index.php?route=product/product&product_id=263Google Scholar
- SLAMTEC. 2018. RPLIDAR Interface Protocol and Application Notes. http://bucket.download.slamtec.com/b42b54878a603e13c76a0a0500b53595846614c6/LR001_SLAMTEC_rplidar_protocol_v1.1_en.pdfGoogle Scholar
- G Smeets. 1977. Laser interference microphone for ultrasonics and nonlinear acoustics. The Journal of the Acoustical Society of America 61, 3 (1977), 872--875.Google ScholarCross Ref
- John R Speciale. 2001. Pulsed laser microphone. US Patent 6,301,034.Google Scholar
- Roborock Technology. 2020. Roborock S5 Max Robot Vacuum & Mop Cleaner. https://us.roborock.com/pages/roborock-s5-maxGoogle Scholar
- Television Music. 2015. Music by David Cebert PBS News Hour Open Theme. https://www.youtube.com/watch?v=gzofWrHsmK4&list=PLtxqRD7TEIMdPsuCVV1fK30zQzdJ9gBtn&index=8&t=0sGoogle Scholar
- Television Music. 2018. The 11th Hour Theme Music - MSNBC. https://www.youtube.com/watch?v=44b5L-vn3_4&list=PLtxqRD7TEIMdPsuCVV1fK30zQzdJ9gBtn&index=3&t=0sGoogle Scholar
- Television Music. 2018. Fox Report Theme Music - Fox News. https://www.youtube.com/watch?v=ov8lAenbJUg&list=PLtxqRD7TEIMdPsuCVV1fK30zQzdJ9gBtn&index=9&t=0sGoogle Scholar
- Television Music. 2018. Hannity Theme Music - Fox News. https://www.youtube.com/watch?v=xyzO0kA0ZX0&list=PLtxqRD7TEIMdPsuCVV1fK30zQzdJ9gBtn&index=4&t=0sGoogle Scholar
- Television Music. 2018. MSNBC Election Result Theme (Loop). https://www.youtube.com/watch?v=vlnKoDzCZxE&list=PLtxqRD7TEIMdPsuCVV1fK30zQzdJ9gBtn&index=7&t=0sGoogle Scholar
- Television Music. 2018. Shepard Smith Reporting Theme Music - Fox News. https://www.youtube.com/watch?v=Khr8wCB-G0U&list=PLtxqRD7TEIMdPsuCVV1fK30zQzdJ9gBtn&index=13&t=0sGoogle Scholar
- Television Music. 2018. The Story Theme Music - Fox News. https://www.youtube.com/watch?v=xak5VCT4_do&list=PLtxqRD7TEIMdPsuCVV1fK30zQzdJ9gBtn&index=11&t=0sGoogle Scholar
- Television Music. 2020. Fox News Democracy 2020 Theme Music. https://www.youtube.com/watch?v=aWk9DypnyPI&list=PLtxqRD7TEIMdPsuCVV1fK30zQzdJ9gBtn&index=5&t=0sGoogle Scholar
- AV Test. 2019. From the Land of Smiles - Xiaomi Roborock S55. https://www.iot-tests.org/2019/02/from-the-land-of-smiles-xiaomi-roborock-s55/Google Scholar
- Michele Valenti, Aleksandr Diment, Giambattista Parascandolo, Stefano Squartini, and Tuomas Virtanen. 2016. DCASE 2016 acoustic scene classification using convolutional neural networks. In Proc. Workshop Detection Classif. Acoust. Scenes Events. 95--99.Google Scholar
- Chen-Chia Wang, Sudhir Trivedi, Feng Jin, V Swaminathan, Ponciano Rodriguez, and Narasimha S Prasad. 2009. High sensitivity pulsed laser vibrometer and its application as a laser microphone. Applied Physics Letters 94, 5 (2009), 051112.Google ScholarCross Ref
- Winston Wang. 2019. Calculating Political Bias and Fighting Partisanship with AI. https://www.thebipartisanpress.com/politics/calculating-political-bias-and-fighting-partisanship-with-ai/Google Scholar
- Teng Wei, Shu Wang, Anfu Zhou, and Xinyu Zhang. 2015. Acoustic eavesdropping through wireless vibrometry. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. 130--141.Google ScholarDigital Library
- Lawrence B. Wolff, Diego A. Socolinsky, and Christopher K. Eveland. 2003. Using infrared sensor technology for face recognition and human identification. In Infrared Technology and Applications XXIX, Bjorn F. Andresen and Gabor F. Fulop (Eds.), Vol. 5074. International Society for Optics and Photonics, SPIE, 757 -- 766. Google ScholarCross Ref
- Chenhan Xu, Zhengxiong Li, Hanbin Zhang, Aditya Singh Rathore, Huining Li, Chen Song, Kun Wang, and Wenyao Xu. 2019. Waveear: Exploring a mmwave-based noise-resistant speech sensing for voice-user interface. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services. 14--26.Google ScholarDigital Library
- YDLIDAR. 2020. YDLIDAR G6. https://www.ydlidar.com/service_support/download.html?gid=6Google Scholar
- Yida. 2020. What is a Time of Flight Sensor and How does a ToF Sensor work? https://www.seeedstudio.com/blog/2020/01/08/what-is-a-time-of-flight-sensor-and-how-does-a-tof-sensor-work/Google Scholar
- Xue Ying. 2019. An overview of Overfitting and its solutions. In Journal of Physics: Conference Series, Vol. 1168. IOP Publishing, 022022.Google Scholar
- Li Zhang, Parth H. Pathak, Muchen Wu, Yixin Zhao, and Prasant Mohapatra. 2015. AccelWord: Energy Efficient Hotword Detection Through Accelerometer (ACM MobiSys). Google ScholarDigital Library
Index Terms
Spying with your robot vacuum cleaner: eavesdropping via lidar sensors
Recommendations
LidarPhone: acoustic eavesdropping using a lidar sensor: poster abstract
SenSys '20: Proceedings of the 18th Conference on Embedded Networked Sensor SystemsPrivate conversations are an attractive target for malicious actors intending to conduct audio eavesdropping attacks. Previous works discovered unexpected vectors for these attacks, such as analyzing high-speed video of objects adjacent to sound sources,...
Keyboard Emanations in Remote Voice Calls: Password Leakage and Noise(less) Masking Defenses
CODASPY '18: Proceedings of the Eighth ACM Conference on Data and Application Security and PrivacyKeyboard acoustic side channel attacks to date have been mostly studied in the context of an adversary eavesdropping on keystrokes by placing a listening device near the intended victim creating a local eavesdropping scenario. However, being in close ...
An Eavesdropping System Based on Magnetic Side-Channel Signals Leaked by Speakers
The use of speakers in electronic devices has become widespread, but the security risks associated with micro-speakers, such as earphones, are often overlooked. Many assume that soundproof barriers can prevent sound leakage and protect privacy. This paper ...
Comments