skip to main content
10.1145/3573428.3573675acmotherconferencesArticle/Chapter ViewAbstractPublication PageseitceConference Proceedingsconference-collections
research-article

Shadow Generation for Composite Image with Multi-level Feature Fusion

Published: 15 March 2023 Publication History

Abstract

Shadow generation for composite image aims to generate shadows for the foreground with reference to the background information, reducing the floating feeling of the foreground due to the lack of shadows caused by simple overlays. The existing two-stage methods are not effective in generating shadows for composite images with complex foregrounds. First, in the Shadow Mask Prediction stage, foreground information and background lighting information are not fully extracted, resulting in inaccurate prediction range and shape of foreground shadows. Second, in the Shadow Filling Stage, the amount of shadow parameter information will decrease as the residual network level deepens, and the shadow parameter prediction is inaccurate, which greatly affects the foreground shadow filling effect. Therefore, we add a multi-scale feature enhancement module to obtain a wider range of feature information and improve the mask prediction accuracy. Meanwhile, we propose a multi-level feature fusion module to reduce the loss of information in the process of shadow parameter prediction by multiplexing features at different levels. Our experiments on the public dataset DESOBA show that the method generates shadow regions with more accurate range and shape.

References

[1]
Hong, Y., Niu, L., & Zhang, J. 2022, June. Shadow generation for composite image in real-world scenes. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 36, No. 1, pp. 914-922).
[2]
Zhang, S., Liang, R., & Wang, M. 2019. Shadowgan: Shadow synthesis for virtual objects with conditional adversarial networks. Computational Visual Media, 5(1), 105-115.
[3]
Liu, D., Long, C., Zhang, H., Yu, H., Dong, X., & Xiao, C. 2020. Arshadowgan: Shadow generative adversarial network for augmented reality in single light scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8139-8148).
[4]
Hu, X., Jiang, Y., Fu, C. W., & Heng, P. A. 2019. Mask-shadowgan: Learning to remove shadows from unpaired data. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 2472-2481).
[5]
Liao, B., Zhu, Y., Liang, C., Luo, F., & Xiao, C. 2019. Illumination animating and editing in a single picture using scene structure estimation. Computers & Graphics, 82, 53-64.
[6]
Liu, B., Xu, K., & Martin, R. R. 2017. Static scene illumination estimation from videos with applications. Journal of Computer Science and Technology, 32(3), 430-442.
[7]
Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1125-1134).
[8]
Karsch, K., Hedau, V., Forsyth, D., & Hoiem, D. 2011. Rendering synthetic objects into legacy photographs. ACM Transactions on Graphics (TOG), 30(6), 1-12.
[9]
Karsch, K., Sunkavalli, K., Hadap, S., Carr, N., Jin, H., Fonte, R., ... & Forsyth, D. 2014. Automatic scene inference for 3d object compositing. ACM Transactions on Graphics (TOG), 33(3), 1-15.
[10]
Wang, T., Hu, X., Wang, Q., Heng, P. A., & Fu, C. W. 2020. Instance shadow detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1880-1889).
[11]
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. 2015. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9).
[12]
Ronneberger, O., Fischer, P., & Brox, T. 2015, October. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.
[13]
He, K., Zhang, X., Ren, S., & Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
[14]
Barron, J. T., & Malik, J. 2014. Shape, illumination, and reflectance from shading. IEEE transactions on pattern analysis and machine intelligence, 37(8), 1670-1687.
[15]
Schieber, T. A., Carpi, L., Díaz-Guilera, A., Pardalos, P. M., Masoller, C., & Ravetti, M. G. 2017. Quantification of network structural dissimilarities. Nature communications, 8(1), 1-10.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
EITCE '22: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering
October 2022
1999 pages
ISBN:9781450397148
DOI:10.1145/3573428
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 March 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Composite image
  2. Multi-level feature fusion
  3. Multi-scale
  4. Shadow generation

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

EITCE 2022

Acceptance Rates

Overall Acceptance Rate 508 of 972 submissions, 52%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 60
    Total Downloads
  • Downloads (Last 12 months)39
  • Downloads (Last 6 weeks)7
Reflects downloads up to 30 Oct 2024

Other Metrics

Citations

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media