Jiyeon Lee
Abstract
Abstract
WebVR is an emerging technology that allows users to experience VR (Virtual Reality) through typical web browsers, providing an integrated environment for various VR applications. One important problem of the VR technology is how to securely interact with users, in particular, implementing secure text input. A promising approach is to use a virtual keyboard rendered as a VR object. The VR user can enter certain text by clicking a sequence of virtual keys through the VR controllers, and the input text is handled in a secure way. However, despite the sensitivity of the input text, we found that there is a critical vulnerability that the VR controllers are not properly protected. The VR controller status can be disclosed to malicious entities, imposing a severe threat that an attacker's website can infer the input text by eavesdropping and analyzing the VR controller's movements. To accurately infer the input, the attacker should address two challenges: 1) determining which clicks correspond to the virtual keyboard and 2) identifying which key is pressed. In this paper, we propose a new keystroke inference attack framework, VRKeyLogger, that addresses such challenges with two key components: key-click classifier and key-click identifier. The key-click classifier effectively distinguishes clicks on the virtual keyboard based on the SVM classifier trained by the major features of the VR controller uses. The key-click identifier then accurately identifies which key is pressed by transforming the clicked position into the local coordinate system of the virtual keyboard. We implemented a proof-of-concept prototype and conducted a user study with nine participants. In the extensive user study with three real-world WebVR applications, our VRKeyLogger results in classification and identification accuracy of 93.98 and 96.8% on average, respectively. This implies that the proposed attack poses a serious threat to WebVR security.
- W3C, 2022a : WebVR 1.1. https://immersive-web.github.io/webvr/spec/1.1/ (last visited: 2022-03-05).Google Scholar
- Ling et al., 2019 ,
I know what you enter on Gear VR , in: 7th IEEE Conference on Communications and Network Security, CNS, IEEE, 2019.Google Scholar - Arafat et al., 2021 ,
VR-Spy: a side-channel attack on virtual key-logging in VR headsets , in: IEEE Virtual Reality and 3D User Interfaces, VR, IEEE, 2021.Google Scholar - A-Frame, 2022a : A WebVR implementation platform. https://aframe.io/docs/0.9.0/introduction/ (last visited: 2022-03-05).Google Scholar
- L. OpenGL, 2022 : Coordinate systems. https://learnopengl.com/Getting-started/Coordinate-Systems (last visited: 2022-03-05).Google Scholar
- Three.js, 2022a : A JavaScript 3D library. https://threejs.org/ (last visited: 2022-03-05).Google Scholar
- M. D. Network, 2022a : Document Object Model (DOM). https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Introduction (last visited: 2022-03-05).Google Scholar
- M. D. Network, 2022b : Inputs and input sources. https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API/Inputs (last visited: 2022-03-05).Google Scholar
- A-Frame, 2022b : A-frame raycaster system. https://aframe.io/docs/1.3.0/components/raycaster.html (last visited: 2022-03-05).Google Scholar
- Three.js, 2022b : Three.js raycaster system. https://threejs.org/docs/index.html#api/en/core/Raycaster (last visited: 2022-03-05).Google Scholar
- GitHub, 2022a : Aframe-keyboard. https://github.com/WandererOU/aframe-keyboard (last visited: 2022-03-05).Google Scholar
- GitHub, 2022b : Three-mesh-ui. https://github.com/felixmariotto/three-mesh-ui (last visited: 2022-03-05).Google Scholar
- GitHub, 2022c : Aframe-super-keyboard. https://github.com/supermedium/aframe-super-keyboard (last visited: 2022-03-05).Google Scholar
- GitHub, 2022d : vr-keyboard. https://github.com/erosmarcon/vr-keyboard (last visited: 2022-03-05).Google Scholar
- M. D. Network, 2022c : GamePad API. https://developer.mozilla.org/en-US/docs/Web/API/Gamepad_API (last visited: 2022-03-05).Google Scholar
- Barth et al., 2009 , Securing Frame Communication in Browsers, 2009.Google Scholar
- M. D. Network, 2022d : iframe: inline frame element. https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe (last visited: 2022-03-05).Google Scholar
- M. D. Network, 2022e : Web socket. https://developer.mozilla.org/en-US/docs/Web/API/WebSocket (last visited: 2022-03-05).Google Scholar
- HTC, 2022 : VIVE pro specs. https://www.vive.com/us/product/vive-pro/ (last visited: 2022-03-05).Google Scholar
- M. VR, 2022a : A-Blast. https://aframe.io/a-blast/ (last visited: 2022-03-05).Google Scholar
- Supermedium, 2022 : Moon rider. https://moonrider.xyz/ (last visited: 2022-03-05).Google Scholar
- M. VR, 2022b : A-Painter. https://aframe.io/a-painter/ (last visited: 2022-03-05).Google Scholar
- Bos et al., 2008 , A theory on visually induced motion sickness, Displays 29 (2) (2008) 47–57.Google Scholar
- W3C, 2022b : WebXR device API. https://www.w3.org/TR/webxr/ (last visited: 2022-03-05).Google Scholar
- Lee et al., 2021 ,
AdCube: WebVR Ad fraud and practical confinement of third-party Ads , in: 30th USENIX Security Symposium, USENIX Security, USENIX Association, 2021.Google Scholar - Luo et al., 2020 ,
OcuLock: exploring human visual system for authentication in virtual reality head-mounted display , in: 27th Annual Network and Distributed System Security Symposium, NDSS, The Internet Society, 2020.Google Scholar - Schneegass et al., 2016 ,
SkullConduct: biometric user identification on eyewear computers using bone conduction through the skull , in: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI, ACM, 2016.Google Scholar - Bianchi and Oakley, 2016 , Wearable authentication: trends and opportunities, IT, Inf. Technol. 58 (5) (2016) 255–262.Google Scholar
- Zhang et al., 2017 ,
AugAuth: shoulder-surfing resistant authentication for augmented reality , in: IEEE International Conference on Communications, ICC, IEEE, 2017.Google Scholar - Liu et al., 2015a ,
Snooping keystrokes with mm-level audio ranging on a single phone , in: Proc. ACM MobiCom, 2015.Google Scholar - Liu et al., 2015b ,
When good becomes evil: keystroke inference with smartwatch , in: Proc. ACM CCS, 2015.Google Scholar - Wang et al., 2015 ,
MoLe: motion leaks through smartwatch sensors , in: Proc. ACM MobiCom, 2015.Google Scholar - Narrain et al., 2014 ,
Single-stroke language-agnostic keylogging using stereo-microphones and domain specific machine learning categories and subject descriptors , in: Proc. ACM WiSec, 2014.Google Scholar - Shumailov et al., 2019 (2019): Hearing your touch: a new acoustic side channel on smartphones. arXiv preprint arXiv:1903.11137.Google Scholar
- Kim et al., 2020 , TapSnoop: leveraging tap sounds to infer tapstrokes on touchscreen devices, IEEE Access 8 (2020) 14737–14748.Google Scholar
- Xu et al., 2011 ,
TapLogger: inferring user inputs on smartphone touchscreens using on-board motion sensors , in: Proc. ACM WiSec, 2011.Google Scholar - Miluzzo et al., 2012 ,
Tapprints: your finger taps have fingerprints , in: Proc. ACM MobiSys, 2012.Google Scholar - Cai and Chen, 2011 ,
TouchLogger: inferring keystrokes on touch screen from smartphone motion , in: Proc. HotSec, 2011.Google Scholar - Ping et al., 2015 ,
TextLogger: inferring longer inputs on touch screen using motion sensors , in: Proc. ACM WiSec, 2015.Google Scholar - Chen et al., 2018 ,
EyeTell: video-assisted touchscreen keystroke inference from eye movements , in: Proc. of IEEE Symposium on Security and Privacy, 2018.Google Scholar - Wang et al., 2018 ,
Inferring password using smartphone front camera , in: Proc. of ACM MobiQuitous, 2018.Google Scholar - Funk et al., 2019 ,
LookUnlock: using spatial-targets for user-authentication on HMDs , in: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, CHI, ACM, 2019.Google Scholar - Mathis et al., 2020 ,
Knowledge-driven biometric authentication in virtual reality , in: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, CHI, ACM, 2020.Google Scholar
Recommendations
Social VR Platform: Building 360-degree Shared VR Spaces
TVX '17 Adjunct: Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online VideoVirtual Reality (VR) and 360-degree video are set to become part of the future social environment, enriching and enhancing the way we share experiences and collaborate remotely. In this demo, we present our ongoing efforts towards social and shared VR; ...
Virtual reality conferencing: multi-user immersive VR experiences on the web
MMSys '18: Proceedings of the 9th ACM Multimedia Systems ConferenceVirtual Reality (VR) and 360-degree video are set to become part of the future social environment, enriching and enhancing the way we share experiences and collaborate remotely. While Social VR applications are getting more momentum, most services ...
Experiencing Virtual Reality Together: Social VR Use Case Study
TVX '18: Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online VideoAs Virtual Reality (VR) applications gain more momentum recently, the social and communication aspects of VR experiences become more relevant. In this paper, we present some initial results of understanding the type of applications and factors that ...
Comments