Spatiotemporal security in mixed reality systems

ABSTRACT

This paper exhaustively explores the threat landscape of coordinated spatiotemporal attacks in mixed reality systems. Novel devicelevel and cross-device time translation and spatial shift attacks are launched, and their impact on deep learning based sensor fusion is evaluated. A major focus of this work is to establish stealthiness in the presence of sophisticated security mechanisms with an added constraint that mixed reality systems allow minimal time durations for covert operation. The efficacy of proposed attacks is evaluated through a preliminary study on inertial and visual data streams.

References

1. P. Casey, I. Baggili, and A. Yarramreddy. 2019. Immersive Virtual Reality Attacks and the Human Joystick. IEEE TDSC (2019).
2. C. Chen, P. Zhao, C.X. Lu, W. Wang, A. Markham, and N. Trigoni. 2018. OxIOD: The Dataset for Deep Inertial Odometry. arXiv:1809.07491 [cs.RO]
3. S. S. Sandha, J. Noor, F. M. Anwar, and M. Srivastava. 2020. Time Awareness in Deep Learning-Based Multimodal Fusion Across Smartphone Platforms. In IEEE/ACM IoTDI.
4. I. Sluganovic, M. Serbec, A. Derek, and I. Martinovic. 2017. HoloPair: Securing Shared Augmented Reality Using Microsoft HoloLens. In ACSAC.
5. Y. Tu, Z. Lin, I. Lee, and X. Hei. 2018. Injected and Delivered: Fabricating Implicit Control over Actuation Systems by Spoofing Inertial Sensors. In USENIX Security.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Mixed / augmented reality