Pietro Lungaro
ABSTRACT
Immersivemote is a novel technology combining our former foveated streaming solution with our novel foveated AI concept. While we have previously shown that foveated streaming can achieve 90% bandwidth savings, as compared to existing streaming solutions, foveated AI is designed to enable real-time video augmentations that are controlled through eye-gaze. The combined solution is therefore capable of effectively interfacing remote operators with mission critical information obtained, in real time, from task-aware machine understanding of the scene and IoT data.
Supplemental Material
- Stephen M. Casner, Edwin L. Hutchins, and Don Norman. 2016. The Challenges of Partially Automated Driving. Commun. ACM 59, 5 (April 2016), 70--77. Google Scholar
Digital Library
- Ericsson. 2017. Remote monitoring and control of vehicles - Ericsson. https://www.ericsson.com/en/mobility-report/remote-monitoring-and-control-of-vehiclesGoogle Scholar
- GSMA. 2018. Intelligent Connectivity: the Fusion of 5G, AI and IoT. https://www.gsma.com/iot/news/intelligent-connectivity-5g-ai-iot/Google Scholar
- Brian Guenter, Mark Finch, Steven Drucker, Desney Tan, and John Snyder. 2012. Foveated 3D Graphics. ACM Trans. Graph. 31, 6 (Nov. 2012), 164:1--164:10. Google ScholarDigital Library
- Pietro Lungaro, Rickard Sjoberg, Alfredo Jose Fanghella Valero, Ashutosh Mittal, and Konrad Tollmar. 2018. Gaze-Aware Streaming Solutions for the Next Generation of Mobile VR Experiences. IEEE Transactions on Visualization and Computer Graphics 24, 4 (April 2018), 1535--1544. Google ScholarDigital Library
- Marvin Minksy. 1980. Telepresence. https://web.media.mit.edu/~minsky/papers/Telepresence.htmlGoogle Scholar
- John Pretlove and Charlotte Skourup. 2007. Human in the loop. ABB Review 1 (2007), 6--10.Google Scholar
- Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement. arXiv (2018).Google Scholar
Index Terms
Immersivemote: combining foveated AI and streaming for immersive remote operations
Recommendations
Latency Requirements for Foveated Rendering in Virtual Reality
Special Issue SAP 2017Foveated rendering is a performance optimization based on the well-known degradation of peripheral visual acuity. It reduces computational costs by showing a high-quality image in the user’s central (foveal) vision and a lower quality image in the ...
Foveated streaming of real-time graphics
MMSys '21: Proceedings of the 12th ACM Multimedia Systems ConferenceRemote rendering systems comprise powerful servers that render graphics on behalf of low-end client devices and stream the graphics as compressed video, enabling high end gaming and Virtual Reality on those devices. One key challenge with them is the ...
Exploiting the GBVS for Saliency aware Gaze Heatmaps
ETRA '20 Short Papers: ACM Symposium on Eye Tracking Research and ApplicationsAnalyzing visual perception in scene images is dominated by two different approaches: 1.) Eye Tracking, which allows us to measure the visual focus directly by mapping a detected fixation to a scene image, and 2.) Saliency maps, which predict the ...
Comments