skip to main content
research-article

Context-Aware Photography Learning for Smart Mobile Devices

Published:21 October 2015Publication History
Skip Abstract Section

Abstract

In this work we have developed a photography model based on machine learning which can assist a user in capturing high quality photographs. As scene composition and camera parameters play a vital role in aesthetics of a captured image, the proposed method addresses the problem of learning photographic composition and camera parameters. Further, we observe that context is an important factor from a photography perspective, we therefore augment the learning with associated contextual information. The proposed method utilizes publicly available photographs along with social media cues and associated metainformation in photography learning. We define context features based on factors such as time, geolocation, environmental conditions and type of image, which have an impact on photography. We also propose the idea of computing the photographic composition basis, eigenrules and baserules, to support our composition learning. The proposed system can be used to provide feedback to the user regarding scene composition and camera parameters while the scene is being captured. It can also recommend position in the frame where people should stand for better composition. Moreover, it also provides camera motion guidance for pan, tilt and zoom to the user for improving scene composition.

Skip Supplemental Material Section

Supplemental Material

References

  1. R. Abdullah, M. Christie, G. Schofield, C. Lino, and P. Olivier. 2011. Advanced composition in virtual camera control. In Proceedings of the International Symposium on Smart Graphics. Springer, 13--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. R. Achanta and S. Susstrunk. 2010. Saliency detection using maximum symmetric surround. In Proceedings of the International Conference on Image Processing. 2653--2656.Google ScholarGoogle Scholar
  4. S. Bae, A. Agarwala, and F. Durand. 2010. Computational rephotography. ACM Trans. Graphics 24:1--24:15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. S. Banerjee and B. L. Evans. 2007. In-camera automation of photographic composition rules. IEEE Trans. Image Process. 1807--1820. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. W. Bares. 2006. A photographic composition assistant for intelligent virtual 3D camera systems. In Proceedings of the International Symposium on Smart Graphics. Springer, 172--183.Google ScholarGoogle ScholarCross RefCross Ref
  7. H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool. 2008. Speeded-Up Robust Features (SURF). In Proceedings of the Conference on Computer Vision and Image Understanding. 346--359. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. S. Bourke, K. McCarthy, and B. Smyth. 2011. The social camera: A case-study in contextual image recommendation. In Proceedings of the 16th International Conference on Intelligent User Interfaces. 13--22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. D. Butterfield, C. Fake, C. Henderson-Begg, and S. Mourachov. 2006. Interestingness ranking of media objects. US Patent App. 11/350,981.Google ScholarGoogle Scholar
  10. C. C. Chang and C. J. Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 27:1--27:27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. B. Cheng, B. Ni, S. Yan, and Q. Tian. 2010. Learning to photograph. In Proceedings of the International Conference on Multimedia. ACM, 291--300. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. N. Dalal and B. Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 886--893. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. R. Statistical Soc. Series B (Methodological), 1--38.Google ScholarGoogle Scholar
  14. Weather Forecast and Reports. 2014. (2014). Retrieved March 3, 2014 from http://www.wunderground.com.Google ScholarGoogle Scholar
  15. M. Freeman. 2007. The Photographer's Eye: Composition and Design for Better Digital Photos. Focal Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. H. Fu, X. Han, and Q. H. Phan. 2013. Data-driven suggestions for portrait posing. In Proceedings of the SIGGRAPH Asia Conference on Emerging Technologies. ACM, 7:1--7:3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. R. Gadde and K. Karlapalem. 2011. Aesthetic guideline driven photography by robots. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence. 2060--2065. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J. Huang, X. Yang, X. Fang, W. Lin, and R. Zhang. 2011. Integrating visual saliency and consistency for re-ranking image search results. IEEE Trans. Multimedia, 653--661. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. R. Jacobson. 2000. The Manual of Photography: Photographic and Digital Imaging. Focal Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. S. Kelby. 2006. The Digital Photography Book. Peachpit Press, Berkeley, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. J. G. Kim, H. S. Chang, J. Kim, and H. M. Kim. 2000. Efficient camera motion characterization for MPEG video indexing. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1171--1174.Google ScholarGoogle Scholar
  22. L. Zheng, Y. Xiaokang, L. Weiyao, Z. Hongyuan, and C. N. Xiaolin. 2014. Inferring user image-search goals under the implicit guidance of users. IEEE Trans. Circuits Syst. Video Technol. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. D. D. Lee and H. S. Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 788--791.Google ScholarGoogle ScholarCross RefCross Ref
  24. C. Li, A. C. Loui, and T. Chen. 2010. Towards aesthetics: A photo quality assessment and photo selection system. In Proceedings of the International Conference on Multimedia. 271--280. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. C. Liu, J. Yuen, and A. Torralba. 2011. Nonparametric scene parsing via label transfer. IEEE Trans. Pattern Anal. Mach. Intell. 2368--2382. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. C. Lujun, Y. Hongxun, S. Xiaoshuai, and Z. Hongming. 2012. Real-time viewfinder composition assessment and recommendation to mobile photographing. In Proceedings of the Pacific-Rim Conference on Advances in Multimedia Information Processing. 707--714. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. S. Ma, Y. Fan, and Chang W. Chen. 2014. Pose maker: A pose recommendation system for person in the landscape photographing. In Proceedings of the ACM International Conference on Multimedia. 1053--1056. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. L. Marchesotti, F. Perronnin, D. Larlus, and G. Csurka. 2011. Assessing the aesthetic quality of photographs using generic image descriptors. In Proceedings of the IEEE International Conference on Computer Vision. 1784--1791. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. H. Mitarai, Y. Itamiya, and A. Yoshitaka. 2013. Interactive photographic shooting assistance based on composition and saliency. In Computational Science and Its Applications, 348--363.Google ScholarGoogle Scholar
  30. B. Ni, M. Xu, B. Cheng, M. Wang, S. Yan, and Q. Tian. 2013. Learning to photograph: A compositional perspective. IEEE Trans. Multimedia 1138--1151. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Y. S. Rawat and M. S. Kankanhalli. 2014. Context-based photography learning using crowdsourced images and social media. In Proceedings of the ACM International Conference on Multimedia, Grand Challenge. 217--220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. H. Su, T. Chen, C. Kao, W. Hsu, and S. Chien. 2012. Preference-aware view recommendation system for scenic photos based on bag-of-aesthetics-preserving features. IEEE Trans. Multimedia. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Sunrise and Sunset Calculator. 2014. http://www.timeanddate.com.Google ScholarGoogle Scholar
  34. M. A. Turk and A. P. Pentland. 1991. Face recognition using eigenfaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 586--591.Google ScholarGoogle Scholar
  35. P. P. Wang, W. Zhang, J. Li, and Y. Zhang. 2008. Online photography assistance by exploring geo-referenced photos on MID / UMPC. In Proceedings of the IEEE 10th Workshop on Multimedia Signal Processing. 6--10.Google ScholarGoogle Scholar
  36. W. Wang, W. Lin, Y. Chen, J. Wu, J. Wang, and B. Sheng. 2014. Finding coherent motions and semantic regions in crowd scenes: A diffusion and clustering approach. In Proceedings of the European Conference on Computer Vision. 756--771.Google ScholarGoogle Scholar
  37. P. Xu, H. Yao, R. Ji, X. M. Liu, and X. Sun. 2014. Where should I stand? Learning based human position recommendation for mobile photographing. Multimedia Tools Appl. 69:3--29. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. W. Yin, T. Mei, and C. W. Chen. 2012. Crowdsourced learning to photograph via mobile devices. In Proceedings of the IEEE International Conference on Multimedia and Expo. 812--817. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. W. Yin, T. Mei, C. W. Chen, and S. Li. 2014. Socialized mobile photography: Learning to photograph with social context via mobile devices. IEEE Trans. Multimedia. 184--200.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Context-Aware Photography Learning for Smart Mobile Devices

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 12, Issue 1s
      Special Issue on Smartphone-Based Interactive Technologies, Systems, and Applications and Special Issue on Extended Best Papers from ACM Multimedia 2014
      October 2015
      317 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/2837676
      Issue’s Table of Contents

      Copyright © 2015 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 21 October 2015
      • Accepted: 1 July 2015
      • Revised: 1 April 2015
      • Received: 1 January 2015
      Published in tomm Volume 12, Issue 1s

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!