Fusing wifi and camera for fast motion tracking and person identification: demo abstract

ABSTRACT

Human sensing, motion tracking, and identification are at the center of numerous applications such as customer analysis, public safety, smart cities, and surveillance. To enable such capabilities, existing solutions mostly rely on vision-based approaches, e.g., facial recognition that is perceived to be too privacy invasive. Other camera-based approaches using body appearances lack long-term re-identification capability. WiFi-based approaches require the installation and maintenance of multiple units. We propose a novel system - called EyeFi [2] - that overcomes these limitations on a standalone device by fusing camera and WiFi data. We use a three-antenna WiFi chipset to measure WiFi Channel State Information (CSI) to estimate the Angle of Arrival (AoA) using a neural network trained with a novel student-teacher model. Then, we perform cross modal (WiFi, camera) trajectory matching to identify individuals using the MAC address of the incoming WiFi packets. We demonstrate our work using real-world data and showcase improvements over traditional optimization-based methods in terms of accuracy and speed.

References

1. [n.d.]. San Francisco Banned Facial Recognition. https://www.nytimes.com/2019/07/01/us/facial-recognition-san-francisco.html.
2. Shiwei Fang, Tamzeed Islam, Sirajum Munir, and Shahriar Nirjon. 2020. EyeFi: Fast Human Identification Through Vision and WiFi-based Trajectory Matching. In 2020 16th International Conference on Distributed Computing in Sensor Systems (DCOSS). IEEE, 59--68.
3. Shiwei Fang, Sirajum Munir, and Shahriar Nirjon. 2020. Dataset: Person Tracking and Identification using Cameras and Wi-Fi Channel State Information (CSI) from Smartphones. In Proceedings of the 3rd Workshop on Data Acquisition To Analysis.
4. Omar Hamdoun, Fabien Moutarde, Bogdan Stanciulescu, and Bruno Steux. 2008. Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences. In 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras. IEEE, 1--6.
5. Nacer Khalil, Driss Benhaddou, Omprakash Gnawali, and Jaspal Subhlok. 2017. Sonicdoor: scaling person identification with ultrasonic sensors by novel modeling of shape, behavior and walking patterns. In Proceedings of the 4th ACM International Conference on Systems for Energy-Efficient Built Environments. ACM, 3.
6. Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, and Sachin Katti. 2015. Spotfi: Decimeter level localization using wifi. In ACM SIGCOMM computer communication review, Vol. 45. ACM, 269--282.

Index Terms

• Computer systems organization

  • Embedded and cyber-physical systems

    • Sensor networks

    • Sensors and actuators

• Human-centered computing

  • Ubiquitous and mobile computing

    • Ubiquitous and mobile computing systems and tools