Abstract
A recent line of research focuses on crowd density estimation from RGB images for a variety of applications, for example, surveillance and traffic flow control. The performance drops dramatically for low-quality images, such as occlusion, or poor light conditions. However, people are equipped with various wireless devices, allowing the received signals to be easily collected at the base station. As such, another line of research utilizes received signals for crowd counting. Nevertheless, received signals offer only information regarding the number of people, while an accurate density map cannot be derived. As unmanned aerial vehicles (UAVs) are now treated as flying base stations and equipped with cameras, we make the first attempt to leverage both RGB images and received signals for crowd density estimation on UAVs. Specifically, we propose a novel network to effectively fuse the RGB images and received signal strength (RSS) information. Moreover, we design a new loss function that considers the uncertainty from RSS and makes the prediction consistent with the received signals. Experimental results show that the proposed method successfully helps break the limit of traditional crowd density estimation methods and achieves state-of-the-art performance. The proposed dataset is released as a public download for future research.
- [1] . 2014. What will 5G be? IEEE Journal on Selected Areas in Communications 32, 6 (2014), 1065–1082.Google Scholar
Cross Ref
- [2] . 2016. Automatic Update of Crowd and Traffic Data Using Device Monitoring. (
Jul 2016).US Patent 9,401,086. Google Scholar - [3] . 1965. Algorithm for computer control of a digital plotter. IBM Systems Journal 4, 1 (1965), 25–30.Google Scholar
Digital Library
- [4] . 2020. Scale aggregation network for accurate and efficient crowd counting. In Proceedings of the European Conference on Computer Vision (ECCV’18), Munich, Germany. Springer, 757–773.Google Scholar
- [5] . 2012. Counting people with low-level features and Bayesian regression. IEEE Transactions on Image Processing 21, 4 (
April 2012), 2160–2177.Google ScholarDigital Library
- [6] . 2020. Crowd counting with crowd attention convolutional neural network. Neurocomputing 382 (2020), 210–220.
DOI: Google ScholarDigital Library
- [7] . 2019. Learning spatial awareness to improve crowd counting. In Proceedings of the IEEE Conference on Computer Vision (ICCV’19), Seoul, Korea (South). 6151–6160.Google Scholar
Cross Ref
- [8] . 2019. Improving the learning of multi-column convolutional neural network for crowd counting. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19) Nice, France. Association for Computing Machinery, 1897–1906.Google Scholar
Digital Library
- [9] . 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, Vol. 1. 886–893.Google Scholar
Digital Library
- [10] . 2016. A trained-once crowd counting method using differential WiFi channel state information. In Proceedings of the 3rd International Workshop on Physical Analytics (WPA’16), Singapore. 37–42.Google Scholar
Digital Library
- [11] . 2012. Pedestrian detection: An evaluation of the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 4 (
April 2012), 743–761.Google ScholarDigital Library
- [12] . 2014. Crowd counting via head detection and motion flow estimation. In Proceedings of the 22nd ACM International Conference on Multimedia (MM’14) Orlando, FL, USA. ACM, 877–880.Google Scholar
Digital Library
- [13] . 2019. SCAR: Spatial-/channel-wise attention regression networks for crowd counting. Neurocomputing 363 (
October 2019), 1–8.Google ScholarDigital Library
- [14] . 2019. DADNet: Dilated-attention-deformable ConvNet for crowd counting. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19) Nice, France. ACM, 1823–1832.Google Scholar
Digital Library
- [15] . 2014. Crowd density estimation for public transport vehicles. In Proceedings of the International Conference on Extending Database Technology/International Conference on Database Theory (EDBT/ICDT’14), Joint Conference, Athens, Greece. CEUR-WS.org, 315–322.Google Scholar
- [16] . 2014. Spatial pyramid pooling in deep convolutional networks for visual recognition. In Proceedings of the European Conference on Computer Vision (ECCV’14), Zurich, Switzerland. Springer, 346–361.Google Scholar
- [17] . 2016. Dense crowd counting from still images with convolutional neural networks. Journal of Visual Communication and Image Representation 38 (2016), 530–539.
DOI: Google ScholarDigital Library
- [18] . 2013. Multi-source multi-scale counting in extremely dense crowd images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’13), Portland, OR, USA. IEEE Computer Society, 2547–2554.Google Scholar
Digital Library
- [19] . 2015. Detecting humans in dense crowds using locally-consistent scale prior and global occlusion reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 10 (2015), 1986–1998.Google Scholar
Digital Library
- [20] . 2020. Attention scaling for crowd counting. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), Seattle, WA, USA. Computer Vision Foundation, 4705–4714.Google Scholar
Cross Ref
- [21] . 2019. A survey of air-to-ground propagation channel modeling for unmanned aerial vehicles. IEEE Communications Surveys Tutorials 21, 3 (2019), 2361–2391.Google Scholar
Cross Ref
- [22] . 2016. Vision-based counting of pedestrians and cyclists. In IEEE Winter Conference on Applications of Computer Vision (WACV’16). IEEE Computer Society, 1–8.
DOI: Google ScholarCross Ref
- [23] . 2019. Data-driven 3D placement of UAV base stations for arbitrarily distributed crowds. In 2019 IEEE Global Communications Conference (GLOBECOM’19) Waikoloa, HI, USA. IEEE, 1–6.
DOI: Google ScholarDigital Library
- [24] . 2020. Trajectory prediction in heterogeneous environment via attended ecology embedding. In Proceedings of the ACM International Conference on Multimedia Virtual Event/Seattle, WA, USA. ACM, 202–210.Google Scholar
Digital Library
- [25] . 2015. Crowded scene analysis: A survey. IEEE Transactions on Circuits and Systems for Video Technology (TCSVT’15) 25, 3 (2015), 367–386.
DOI: Google ScholarDigital Library
- [26] . 2018. CSRNet: Dilated convolutional neural networks for understanding the highly congested scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18) Salt Lake City, UT, USA. IEEE Computer Society, 1091–1100.Google Scholar
Cross Ref
- [27] . 2019. Density map regression guided detection network for RGB-D crowd counting and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19), Long Beach, CA, USA. Computer Vision Foundation, 1821–1830.Google Scholar
Cross Ref
- [28] . 2020. Bidirectional attention-recognition model for fine-grained object classification. IEEE Transactions on Multimedia 22, 7 (2020), 1785–1795.
DOI: Google ScholarCross Ref
- [29] . 2019. Dense dilated convolutions merging network for semantic mapping of remote sensing images. In Proceedings of Joint Urban Remote Sensing Event (JURSE’19) Vannes, France. IEEE, 1–4.Google Scholar
Cross Ref
- [30] . 2019. Geometric and physical constraints for drone-based head plane crowd density estimation. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’19), Macau, SAR, China. IEEE, 244–249.Google Scholar
- [31] . 2019. Context-aware crowd counting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19), Long Beach, CA, USA. Computer Vision Foundation, 5094–5103.Google Scholar
Cross Ref
- [32] . 2020. Adaptive mixture regression network with local counting map for crowd counting. In Proceedings of the European Conference on Computer Vision (ECCV’20), Glasgow, UK. Springer, 241–257.Google Scholar
Digital Library
- [33] . 2020. Semi-supervised crowd counting via self-training on surrogate tasks. In Proceedings of the European Conference on Computer Vision (ECCV’20), Glasgow, UK. Springer, 242–259.Google Scholar
Digital Library
- [34] . 2015. Fully Convolutional Networks for Semantic Segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15), Boston, MA, USA. IEEE Computer Society, 3431–3440.Google Scholar
- [35] . 2021. Spatiotemporal dilated convolution with uncertain matching for video-based crowd estimation. IEEE Transactions on Multimedia (2021), 1–1.
DOI: Google ScholarDigital Library
- [36] . 2013. Crossing the line: Crowd counting by integer programming with local features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’13), Portland, OR, USA. IEEE Computer Society, 2539–2546.Google Scholar
Digital Library
- [37] . 2016. Measuring the accuracy of crowd counting using WiFi probe-request-frame counting technique. Journal of Telecommunication, Electronic and Computer Engineering 8, 2 (2016), 79–81.Google Scholar
- [38] . 2017. Spatial As Deep: Spatial CNN for Traffic Scene Understanding. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’18), New Orleans, Louisiana, USA. AAAI Press, 7276–7283.Google Scholar
- [39] . 2009. Crowd counting using multiple local features. In Proceedings of the Digital Image Computing: Techniques and Applications (DICTA’09) Melbourne, Australia. IEEE Computer Society, 81–88.Google Scholar
Digital Library
- [40] . 2017. Switching convolutional neural network for crowd counting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17) Honolulu, HI, USA. IEEE Computer Society, 4031–4039.Google Scholar
Cross Ref
- [41] . 2019. Revisiting perspective information for efficient crowd counting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19). Long Beach, CA, USA. Computer Vision Foundation, 7279–7288.Google Scholar
Cross Ref
- [42] . 2019. People crowd density estimation system using deep learning for radio wave sensing of cellular communication. In Proceedings of the International Conference on Artificial Intelligence in Information and Communication (ICAIIC’19) Okinawa, Japan. IEEE, 143–148.Google Scholar
Cross Ref
- [43] . 2020. Learning to count in the crowd from limited labeled data. In Proceedings of the European Conference on Computer Vision (ECCV’20), Glasgow, UK. Springer, 212–229.Google Scholar
Digital Library
- [44] . 2020. S2SiamFC: Self-supervised fully convolutional Siamese network for visual tracking. In Proceedings of the ACM International Conference on Multimedia, Virtual Event / Seattle, WA, USA. ACM, 1948–1957.Google Scholar
Digital Library
- [45] . 2016. End-to-end people detection in crowded scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16), Las Vegas, NV, USA. IEEE Computer Society, 2325–2333.Google Scholar
- [46] . 2017. Principles of Mobile Communication (4th ed.). Springer, Cham.Google Scholar
Cross Ref
- [47] . 2019. Crowd counting via multi-layer regression. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19), Nice, France. ACM, 1907–1915.Google Scholar
Digital Library
- [48] . 2019. PaDNet: Pan-density crowd counting. IEEE Transactions on Image Processing 29 (
November 2019), 2714–2727.Google ScholarCross Ref
- [49] . 2019. Deployment algorithms of flying base stations: 5G and beyond with UAVs. In IEEE Internet of Things Journal 6, 6 (2019), 10009–10027.Google Scholar
- [50] . 2019. Learning from synthetic data for crowd counting in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19), Long Beach, CA, USA. Computer Vision Foundation, 8198–8207.Google Scholar
Cross Ref
- [51] . 2019. Multi-dilation network for crowd counting. In Proceedings of the ACM Multimedia Asia (MMAsia’19) Beijing, China. Association for Computing Machinery, Article 56, 1–6.Google Scholar
Digital Library
- [52] . 2020. Human-like traffic scene understanding system: A survey. IEEE Industrial Electronics Magazine 15, 1 (2020), 6–15.Google Scholar
- [53] . 2018. Capacity enhancement for 5G networks using MmWave aerial base stations: Self-organizing architecture and approach. IEEE Wireless Communications 25, 4 (
August 2018), 58–64.Google ScholarCross Ref
- [54] . 2020. Multi-scale supervised attentive encoder-decoder network for crowd counting. ACM Transactions on Multimedia Computing, Communications, and Applications Article 28, 16, 1 (
April 2020).Google Scholar - [55] . 2020. Unmanned Aerial Vehicle Applications Over Cellular Networks for 5G and Beyond. Springer.Google Scholar
Cross Ref
- [56] . 2016. Single-image crowd counting via multi-column convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16) Las Vegas, NV, USA. IEEE, 589–597.Google Scholar
Cross Ref
- [57] . 2015. Crowd counting in public video surveillance by label distribution learning. Neurocomputing 166 (
Oct. 2015), 151–163.Google ScholarDigital Library
- [58] . 2020. Device-free crowd counting with WiFi channel state information and deep neural networks. Wireless Networks 26, 5 (2020), 3495–3506.
DOI: Google ScholarDigital Library
- [59] . 2020. Vision Meets Drones: Past, Present and Future. (2020).
arxiv:2001.06303 Google Scholar
Index Terms
Improving Crowd Density Estimation by Fusing Aerial Images and Radio Signals
Recommendations
Integral Backstepping Control of an Unconventional Dual-Fan Unmanned Aerial Vehicle
In this paper, a dynamic model of vertical take-off and landing (VTOL) aerial vehicles, having lateral and longitudinal rotor tilting mechanism, is first developed using a Newton---Euler formulation. Then an integral backstepping (IB) control technique ...
An evaluative review of the VTOL technologies for unmanned and manned aerial vehicles
AbstractVTOL (Vertical Take-Off and Landing) capabilities are desired features of both UAVs (Unmanned Aerial Vehicles) and MAVs (Manned Aerial Vehicles) on condition that a comparable flight performance is achieved. VTOL is not only a very ...
Unmanned aerial vehicles
As unmanned aerial vehicles have become more affordable, their popularity with the general public and commercial organisations has seen significant growth in recent years. Whilst remaining a device for both the hobbyist and aircraft-enthusiast to enjoy, ...






Comments