Abstract
Points of interest are an important requirement for location-based services, yet they are editorially curated and maintained, either professionally or through community. Beyond the laborious manual annotation task, further complications arise as points of interest may appear, relocate, or disappear over time, and may be relevant only to specific communities. To assist, complement, or even replace manual annotation, we propose a novel method for the automatic localization of points of interest depicted in photos taken by people across the world. Our technique exploits the geographic coordinates and the compass direction supplied by modern cameras, while accounting for possible measurement errors due to the variability in accuracy of the sensors that produced them. We statistically demonstrate that our method significantly outperforms techniques from the research literature on the task of estimating the geographic coordinates and geographic footprints of points of interest in various cities, even when photos are involved in the estimation process that do not show the point of interest at all.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Finding Social Points of Interest from Georeferenced and Oriented Online Photographs
- S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski. 2009. Building Rome in a day. In Proceedings of the IEEE International Conference on Computer Vision. 72--79.Google Scholar
- S. Ahern, M. Naaman, R. Nair, and J. Yang. 2007. World explorer: Visualizing aggregate data from unstructured text in geo-referenced collections. In Proceedings of the ACM/IEEE-CS Joint Conference on Digital Libraries. 1--10. Google Scholar
Digital Library
- T. Cham, A. Ciptadi, W. Tan, M. Pham, and L. Chia. 2010. Estimating camera pose from a single urban ground-view omnidirectional image and a 2D building outline map. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 366--373.Google Scholar
- F. Chang, C. Chen, and C. Lu. 2004. A linear-time component-labeling algorithm using contour tracing technique. Computer Vision and Image Understanding 93, 2 (2004), 206--220. Google Scholar
Digital Library
- L. Chen and A. Roy. 2009. Event detection from Flickr data through wavelet-based spatial analysis. In Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM’09), 523--532. Google Scholar
Digital Library
- D. J. Crandall, L. Backstrom, D. Huttenlocher, and J. Kleinberg. 2009. Mapping the world’s photos. In Proceedings of the IW3C2 International Conference on World Wide Web. 761--770. Google Scholar
Digital Library
- D. J. Crandall, A. Owens, N. Snavely, and D. P. Huttenlocher. 2013. SfM with MRFs: Discrete-continuous optimization for large-scale structure from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 12 (2013), 2841--2853. Google Scholar
Digital Library
- P. de Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein. 2005. A tutorial on the cross-entropy method. Annals of Operations Research 134, 1 (2005), 19--67.Google Scholar
- M. De Choudhury, M. Feldman, S. Amer-Yahia, N. Golbandi, R. Lempel, and C. Yu. 2010. Automatic construction of travel itineraries using social breadcrumbs. In Proceedings of the ACM Conference on Hypertext and Hypermedia. 35--44. Google Scholar
Digital Library
- A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B 39, 1 (1977), 1--38.Google Scholar
Cross Ref
- C. Doersch, S. Singh, A. Gupta, J. Sivic, and A. A. Efros. 2012. What makes Paris look like Paris? ACM Transactions on Graphics 31, 4 (2012), 1--9. Google Scholar
Digital Library
- C. Dwork, R. Kumar, M. Naor, and D. Sivakumar. 2001. Rank aggregation methods for the Web. In Proceedings of the IW3C2 International Conference on World Wide Web. 613--622. Google Scholar
Digital Library
- B. Epshtein, E. Ofek, Y. Wexler, and P. Zhang. 2007. Hierarchical photo organization using geo-relevance. In Proceedings of the ACM International Conference on Geographic Information Systems. Google Scholar
Digital Library
- M. Ester, H. Kriegel, J. Sander, and X. Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the AAAI International Conference on Knowledge Discovery and Data Mining. 226--231.Google Scholar
- Facebook, Ericsson, and Qualcomm. 2013. A Focus on Efficiency. Technical Report.Google Scholar
- G. Field. 1845. Chromatics; or, the Analogy, Harmony, and Philosophy of Colours. D. Bogue.Google Scholar
- J. Hao, G. Wang, B. Seo, and R. Zimmermann. 2014. Point of interest detection and visual distance estimation for sensor-rich video. IEEE Transactions on Multimedia 16, 7 (2014), 1929--1941.Google Scholar
Cross Ref
- J. Hays and A. A. Efros. 2008. IM2GPS: Estimating geographic information from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.Google Scholar
- J. Heinly, J. L. Schönberger, E. Dunn, and J. Frahm. 2015. Reconstructing the world in six days. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3287--3295.Google Scholar
- L. Hollenstein and R. S. Purves. 2010. Exploring place through user-generated content: Using Flickr tags to describe city cores. Journal of Spatial Information Science 1 (2010), 21--48.Google Scholar
- M. Hölzl, R. Neumeier, and G. Ostzermayer. 2013. Analysis of compass sensor accuracy on several mobile devices in an industrial environment. In Proceedings of the International Conference on Computer Aided Systems Theory. 381--389.Google Scholar
- A. Jaffe, M. Naaman, T. Tassa, and M. Davis. 2006. Generating summaries and visualization for large collections of geo-referenced photographs. In Proceecings of the ACM International Workshop on Multimedia Information Retrieval. 89--98. Google Scholar
Digital Library
- Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. B. Girshick, S. Guadarrama, and T. Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the ACM International Conference on Multimedia. 675--678. Google Scholar
Digital Library
- R. S. Kaminsky, N. Snavely, S. M. Seitz, and R. Szeliski. 2009. Alignment of 3D point clouds to overhead images. In Proceedings of the IEEE Workshop on Computer Vision and Pattern Recognition. 63--70.Google Scholar
- C. F. F. Karney. 2013. Algorithms for geodesics. Journal of Geodesy 87, 1 (2013), 43--55.Google Scholar
Cross Ref
- L. S. Kennedy and M. Naaman. 2008. Generating diverse and representative image search results for landmarks. In Proceedings of the IW3C2 International Conference on World Wide Web. 297--306. Google Scholar
Digital Library
- S. Kisilevich, F. Mansman, and D. A. Keim. 2010. P-DBSCAN: A density based clustering algorithm for exploration and analysis of attractive areas using collections of geo-tagged photos. In Proceedings of the International Conference and Exhibition on Computing for Geospatial Research & Application. Google Scholar
Digital Library
- J. Kosecká and W. Zhang. 2002. Video compass. In Proceedings of the European Conference on Computer Vision. 476--490. Google Scholar
Digital Library
- A. Krizhevsky, I. Sutskever, and G. E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of the Annual Conference on Advances in Neural Information Processing Systems. 1097--1105.Google Scholar
- Y. A. Lacerda, R. G. F. Feitosa, G. A. R. M. Esmeraldo, C. de Souza Baptista, and L. B. Marinho. 2012. Compass clustering: A new clustering method for detection of points of interest using personal collections of georeferenced and oriented photographs. In Proceedings of the Brazilian Symposium on Multimedia and the Web. 281--288. Google Scholar
Digital Library
- Y. Li, D. J. Crandall, and D. P. Huttenlocher. 2009. Landmark classification in large-scale image collections. In Proceedings of the IEEE International Conference on Computer Vision. 1957--1964.Google Scholar
- J. Luo, D. Joshi, J. Yu, and A. C. Gallagher. 2011. Geotagging in multimedia and computer vision—A survey. Multimedia Tools and Applications 51, 1 (2011), 187--211. Google Scholar
Digital Library
- Z. Luo, H. Li, J. Tang, R. Hong, and T. Chua. 2010. Estimating poses of world’s photos with geographic metadata. In Advances in Multimedia Modeling. 695--700. Google Scholar
Digital Library
- L. Mummidi and J. Krumm. 2008. Discovering points of interest from users’ map annotations. GeoJournal. 72, 3--4 (2008), 215--227.Google Scholar
Cross Ref
- M. Naaman, Y. J. Song, A. Paepcke, and H. Garcia-Molina. 2004. Automatic organization for digital photographs with geographic coordinates. In Proceedings of the ACM/IEEE-CS Joint Conference on Digital Libraries. 53--62. Google Scholar
Digital Library
- L. Ojeda and J. Borenstein. 2000. Experimental results with the KVH C-100 fluxgate compass in mobile robots. In Proceedings of the IASTED International Conference on Robotics and Applications.Google Scholar
- J. Paek, J. Kim, and R. Govindan. 2010. Energy-efficient rate-adaptive GPS-based positioning for smartphones. In Proceedings of the ACM International Conference on Mobile Systems, Applications, and Services. 299--314. Google Scholar
Digital Library
- S. Papadopoulos, C. Zigkolis, Y. Kompatsiaris, and A. Vakali. 2011. Cluster-based landmark and event detection for tagged photo collections. IEEE Multimedia 18, 1 (2011), 52--63. Google Scholar
Digital Library
- D. Pelleg and A. W. Moore. 2000. X-means: Extending K-means with efficient estimation of the number of clusters. In Proceedings of the International Conference on Machine Learning. 727--734. Google Scholar
Digital Library
- V. Pihur, S. Datta, and S. Datta. 2007. Weighted rank aggregation of cluster validation measures: A Monte Carlo cross-entropy approach. Bioinformatics 23, 13 (2007), 1607--1615. Google Scholar
Digital Library
- A. Popescu. 2013. CEA LIST’s participation at MediaEval 2013 placing task. In Working Notes of the MediaEval Benchmarking Initiative for Multimedia Evaluation.Google Scholar
- A. Popescu, G. Grefenstette, and P. Moëllic. 2009. Mining tourist information from user-supplied collections. In ACM International Conference on Knowledge and Information Management. 1713--1716. Google Scholar
Digital Library
- A. Popescu and A. Shabou. 2013. Towards precise POI localization with social media. In Proceedings of the ACM International Conference on Multimedia. 573--576. Google Scholar
Digital Library
- T. Quack, B. Leibe, and L. J. Van Gool. 2008. World-scale mining of objects and events from community photo collections. In Proceedings of the ACM Conference on Image and Video Retrieval. 47--56. Google Scholar
Digital Library
- A. Rae, V. Murdock, A. Popescu, and H. Bouchard. 2012. Mining the web for points of interest. In Proceedings of the ACM Interational Conference on Research and Development in Information Retrieval. 711--720. Google Scholar
Digital Library
- R. Raguram, C. Wu, J. Frahm, and S. Lazebnik. 2011. Modeling and recognition of landmark image collections using iconic scene graphs. International Journal of Computer Vision 95, 3 (2011), 213--239. Google Scholar
Digital Library
- T. Rattenbury, N. Good, and M. Naaman. 2007. Towards automatic extraction of event and place semantics from Flickr tags. In Proceedings of the ACM International Conference on Research and Development in Information Retrieval. 103--110. Google Scholar
Digital Library
- T. Rattenbury and M. Naaman. 2009. Methods for extracting place semantics from Flickr tags. ACM Transactions on the Web 3, 1 (2009). Google Scholar
Digital Library
- R. Y. Rubinstein and D. P. Kroese. 2005. The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning. Springer. Google Scholar
Digital Library
- S. Rudinac, A. Hanjalic, and M. Larson. 2011. Finding representative and diverse community contributed images to create visual summaries of geographic areas. In Proceedings of the ACM International Conference on Multimedia. 1109--1112. Google Scholar
Digital Library
- P. Serdyukov, V. Murdock, and R. van Zwol. 2009. Placing Flickr photos on a map. In Proceedings of the ACM International Conference on Research and Development in Information Retrieval. 484--491. Google Scholar
Digital Library
- J. T. Smith. 1797. Remarks on Rural Scenery, with Twenty Etchings of Cottages, from Nature; and Some Observations and Precepts Relative to the Picturesque. N. Smith and I. T. Smith.Google Scholar
- N. Snavely, S. M. Seitz, and R. Szeliski. 2006. Photo tourism: Exploring photo collections in 3D. ACM Transactions on Graphics 25, 3 (2006), 835--846. Google Scholar
Digital Library
- N. Snavely, S. M. Seitz, and R. Szeliski. 2008. Modeling the world from internet photo collections. International Journal of Computer Vision 80, 2 (2008), 189--210. Google Scholar
Digital Library
- M. A. Stephens. 1974. EDF statistics for goodness of fit and some comparisons. Journal of the American Statistical Association 69, 347 (1974), 730--737.Google Scholar
Cross Ref
- H. C. Thode. 2002. Testing for Normality. CRC Press.Google Scholar
- B. Thomee. 2013. Localization of points of interest from georeferenced and oriented photographs. In Proceedings of the ACM Interntional Workshop on Geotagging and Its Applications in Multimedia. 19--24. Google Scholar
Digital Library
- O. Van Laere, S. Schockaert, and B. Dhoedt. 2010. Towards automated georeferencing of Flickr photos. In Proceedings of the ACM Workshop on Geographic Information Retrieval. Google Scholar
Digital Library
- C. Wang, K. Wilson, and N. Snavely. 2013a. Accurate georegistration of point clouds using geographic data. In Proceedings of the International Conference on 3D Vision. 33--40. Google Scholar
Digital Library
- G. Wang, Y. Yin, B. Seo, R. Zimmermann, and Z. Shen. 2013b. Orientation data correction with georeferenced mobile videos. In Proceedings of the ACM International Conference on Advances in Geographic Information Systems. 390--393. Google Scholar
Digital Library
- C. Wu. 2013. Towards linear-time incremental structure from motion. In Proceedings of the IEEE International Conference on 3D Vision. 127--134. Google Scholar
Digital Library
- Y. Yang, Z. Gong, and L. Hou. 2011. Identifying points of interest by self-tuning clustering. In Proceedings of the ACM International Conference on Research and Development in Information Retrieval. 883--892. Google Scholar
Digital Library
- P. A. Zandbergen. 2008. Positional accuracy of spatial data: Non-normal distributions and a critique of the national standard for spatial data accuracy. Transactions in GIS 12, 1 (2008), 103--130.Google Scholar
Cross Ref
- P. A. Zandbergen and S. J. Barbeau. 2011. Positional accuracy of assisted GPS data from high-sensitivity GPS-enabled mobile phone. Journal of Navigation 64, 3 (2011), 381--399.Google Scholar
Cross Ref
- Y. Zheng, Z. Zha, and T. S. Chua. 2011. Research and applications on georeferenced multimedia: A survey. Multimedia Tools and Applications 51, 1 (2011), 77--98. Google Scholar
Digital Library
- Y. Zheng, M. Zhao, Y. Song, H. Adam, U. Buddemeier, A. Bissacco, F. Brucher, T. S. Chua, and H. Neven. 2009. Tour the world: Building a web-scale landmark recognition engine. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1085--1092.Google Scholar
Index Terms
Finding Social Points of Interest from Georeferenced and Oriented Online Photographs
Recommendations
Localization of points of interest from georeferenced and oriented photographs
GeoMM '13: Proceedings of the 2nd ACM international workshop on Geotagging and its applications in multimediaThe geographic coordinates automatically assigned to photos by the built-in GPS devices of cameras register the location a photo was taken at, rather than the location of the main point of interest within the depicted scene. While scientific advances ...
Exploiting photo location and direction for clustering-based points-of-interest discovery
SAC '17: Proceedings of the Symposium on Applied ComputingSeveral works have exploited the geographic information of photos through spatial clustering algorithms aiming at the automatic discovery of points of interest (POIs). The assumption is that dense regions in terms of geographically nearby photos are ...
Mining the web for points of interest
SIGIR '12: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrievalA point of interest (POI) is a focused geographic entity such as a landmark, a school, an historical building, or a business. Points of interest are the basis for most of the data supporting location-based applications. In this paper we propose to ...






Comments