Abstract
In this work we have developed a photography model based on machine learning which can assist a user in capturing high quality photographs. As scene composition and camera parameters play a vital role in aesthetics of a captured image, the proposed method addresses the problem of learning photographic composition and camera parameters. Further, we observe that context is an important factor from a photography perspective, we therefore augment the learning with associated contextual information. The proposed method utilizes publicly available photographs along with social media cues and associated metainformation in photography learning. We define context features based on factors such as time, geolocation, environmental conditions and type of image, which have an impact on photography. We also propose the idea of computing the photographic composition basis, eigenrules and baserules, to support our composition learning. The proposed system can be used to provide feedback to the user regarding scene composition and camera parameters while the scene is being captured. It can also recommend position in the frame where people should stand for better composition. Moreover, it also provides camera motion guidance for pan, tilt and zoom to the user for improving scene composition.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Context-Aware Photography Learning for Smart Mobile Devices
- R. Abdullah, M. Christie, G. Schofield, C. Lino, and P. Olivier. 2011. Advanced composition in virtual camera control. In Proceedings of the International Symposium on Smart Graphics. Springer, 13--24. Google Scholar
Digital Library
- R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. Google Scholar
Digital Library
- R. Achanta and S. Susstrunk. 2010. Saliency detection using maximum symmetric surround. In Proceedings of the International Conference on Image Processing. 2653--2656.Google Scholar
- S. Bae, A. Agarwala, and F. Durand. 2010. Computational rephotography. ACM Trans. Graphics 24:1--24:15. Google Scholar
Digital Library
- S. Banerjee and B. L. Evans. 2007. In-camera automation of photographic composition rules. IEEE Trans. Image Process. 1807--1820. Google Scholar
Digital Library
- W. Bares. 2006. A photographic composition assistant for intelligent virtual 3D camera systems. In Proceedings of the International Symposium on Smart Graphics. Springer, 172--183.Google Scholar
Cross Ref
- H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool. 2008. Speeded-Up Robust Features (SURF). In Proceedings of the Conference on Computer Vision and Image Understanding. 346--359. Google Scholar
Digital Library
- S. Bourke, K. McCarthy, and B. Smyth. 2011. The social camera: A case-study in contextual image recommendation. In Proceedings of the 16th International Conference on Intelligent User Interfaces. 13--22. Google Scholar
Digital Library
- D. Butterfield, C. Fake, C. Henderson-Begg, and S. Mourachov. 2006. Interestingness ranking of media objects. US Patent App. 11/350,981.Google Scholar
- C. C. Chang and C. J. Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 27:1--27:27. Google Scholar
Digital Library
- B. Cheng, B. Ni, S. Yan, and Q. Tian. 2010. Learning to photograph. In Proceedings of the International Conference on Multimedia. ACM, 291--300. Google Scholar
Digital Library
- N. Dalal and B. Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 886--893. Google Scholar
Digital Library
- A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. R. Statistical Soc. Series B (Methodological), 1--38.Google Scholar
- Weather Forecast and Reports. 2014. (2014). Retrieved March 3, 2014 from http://www.wunderground.com.Google Scholar
- M. Freeman. 2007. The Photographer's Eye: Composition and Design for Better Digital Photos. Focal Press. Google Scholar
Digital Library
- H. Fu, X. Han, and Q. H. Phan. 2013. Data-driven suggestions for portrait posing. In Proceedings of the SIGGRAPH Asia Conference on Emerging Technologies. ACM, 7:1--7:3. Google Scholar
Digital Library
- R. Gadde and K. Karlapalem. 2011. Aesthetic guideline driven photography by robots. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence. 2060--2065. Google Scholar
Digital Library
- J. Huang, X. Yang, X. Fang, W. Lin, and R. Zhang. 2011. Integrating visual saliency and consistency for re-ranking image search results. IEEE Trans. Multimedia, 653--661. Google Scholar
Digital Library
- R. Jacobson. 2000. The Manual of Photography: Photographic and Digital Imaging. Focal Press. Google Scholar
Digital Library
- S. Kelby. 2006. The Digital Photography Book. Peachpit Press, Berkeley, CA. Google Scholar
Digital Library
- J. G. Kim, H. S. Chang, J. Kim, and H. M. Kim. 2000. Efficient camera motion characterization for MPEG video indexing. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1171--1174.Google Scholar
- L. Zheng, Y. Xiaokang, L. Weiyao, Z. Hongyuan, and C. N. Xiaolin. 2014. Inferring user image-search goals under the implicit guidance of users. IEEE Trans. Circuits Syst. Video Technol. Google Scholar
Digital Library
- D. D. Lee and H. S. Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 788--791.Google Scholar
Cross Ref
- C. Li, A. C. Loui, and T. Chen. 2010. Towards aesthetics: A photo quality assessment and photo selection system. In Proceedings of the International Conference on Multimedia. 271--280. Google Scholar
Digital Library
- C. Liu, J. Yuen, and A. Torralba. 2011. Nonparametric scene parsing via label transfer. IEEE Trans. Pattern Anal. Mach. Intell. 2368--2382. Google Scholar
Digital Library
- C. Lujun, Y. Hongxun, S. Xiaoshuai, and Z. Hongming. 2012. Real-time viewfinder composition assessment and recommendation to mobile photographing. In Proceedings of the Pacific-Rim Conference on Advances in Multimedia Information Processing. 707--714. Google Scholar
Digital Library
- S. Ma, Y. Fan, and Chang W. Chen. 2014. Pose maker: A pose recommendation system for person in the landscape photographing. In Proceedings of the ACM International Conference on Multimedia. 1053--1056. Google Scholar
Digital Library
- L. Marchesotti, F. Perronnin, D. Larlus, and G. Csurka. 2011. Assessing the aesthetic quality of photographs using generic image descriptors. In Proceedings of the IEEE International Conference on Computer Vision. 1784--1791. Google Scholar
Digital Library
- H. Mitarai, Y. Itamiya, and A. Yoshitaka. 2013. Interactive photographic shooting assistance based on composition and saliency. In Computational Science and Its Applications, 348--363.Google Scholar
- B. Ni, M. Xu, B. Cheng, M. Wang, S. Yan, and Q. Tian. 2013. Learning to photograph: A compositional perspective. IEEE Trans. Multimedia 1138--1151. Google Scholar
Digital Library
- Y. S. Rawat and M. S. Kankanhalli. 2014. Context-based photography learning using crowdsourced images and social media. In Proceedings of the ACM International Conference on Multimedia, Grand Challenge. 217--220. Google Scholar
Digital Library
- H. Su, T. Chen, C. Kao, W. Hsu, and S. Chien. 2012. Preference-aware view recommendation system for scenic photos based on bag-of-aesthetics-preserving features. IEEE Trans. Multimedia. Google Scholar
Digital Library
- Sunrise and Sunset Calculator. 2014. http://www.timeanddate.com.Google Scholar
- M. A. Turk and A. P. Pentland. 1991. Face recognition using eigenfaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 586--591.Google Scholar
- P. P. Wang, W. Zhang, J. Li, and Y. Zhang. 2008. Online photography assistance by exploring geo-referenced photos on MID / UMPC. In Proceedings of the IEEE 10th Workshop on Multimedia Signal Processing. 6--10.Google Scholar
- W. Wang, W. Lin, Y. Chen, J. Wu, J. Wang, and B. Sheng. 2014. Finding coherent motions and semantic regions in crowd scenes: A diffusion and clustering approach. In Proceedings of the European Conference on Computer Vision. 756--771.Google Scholar
- P. Xu, H. Yao, R. Ji, X. M. Liu, and X. Sun. 2014. Where should I stand? Learning based human position recommendation for mobile photographing. Multimedia Tools Appl. 69:3--29. Google Scholar
Digital Library
- W. Yin, T. Mei, and C. W. Chen. 2012. Crowdsourced learning to photograph via mobile devices. In Proceedings of the IEEE International Conference on Multimedia and Expo. 812--817. Google Scholar
Digital Library
- W. Yin, T. Mei, C. W. Chen, and S. Li. 2014. Socialized mobile photography: Learning to photograph with social context via mobile devices. IEEE Trans. Multimedia. 184--200.Google Scholar
Cross Ref
Index Terms
Context-Aware Photography Learning for Smart Mobile Devices
Recommendations
Real-Time Assistance in Multimedia Capture Using Social Media
MM '15: Proceedings of the 23rd ACM international conference on MultimediaIn the last decade, we have seen significant improvement in the ease and cost of capturing multimedia content. However, the aesthetic quality of the content captured by an amateur user still needs substantial improvement. This doctoral research aims at ...
Context-Based Photography Learning using Crowdsourced Images and Social Media
MM '14: Proceedings of the 22nd ACM international conference on MultimediaThis paper presents a photography model based on machine learning which utilizes crowd-sourced images along with social media cues. As scene composition and camera parameters play a vital role in aesthetics of a captured image, the proposed system ...
Metric calibration of a stereo rig
VSR '95: Proceedings of the IEEE Workshop on Representation of Visual ScenesDescribes a method to determine affine and metric calibration for a stereo rig. The method does not involve the use of calibration objects or special motions, but simply a single general motion of the rig with fixed parameters (i.e. camera parameters ...






Comments