Orienting point clouds with dipole propagation

Abstract

Establishing a consistent normal orientation for point clouds is a notoriously difficult problem in geometry processing, requiring attention to both local and global shape characteristics. The normal direction of a point is a function of the local surface neighborhood; yet, point clouds do not disclose the full underlying surface structure. Even assuming known geodesic proximity, calculating a consistent normal orientation requires the global context. In this work, we introduce a novel approach for establishing a globally consistent normal orientation for point clouds. Our solution separates the local and global components into two different sub-problems. In the local phase, we train a neural network to learn a coherent normal direction per patch (i.e., consistently oriented normals within a single patch). In the global phase, we propagate the orientation across all coherent patches using a dipole propagation. Our dipole propagation decides to orient each patch using the electric field defined by all previously orientated patches. This gives rise to a global propagation that is stable, as well as being robust to nearby surfaces, holes, sharp features and noise.

References

1. Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. 2018. Learning representations and generative models for 3d point clouds. In International conference on machine learning. PMLR, 40--49.
2. Gavin Barill, Neil Dickson, Ryan Schmidt, David I.W. Levin, and Alec Jacobson. 2018. Fast Winding Numbers for Soups and Clouds. ACM Transactions on Graphics (2018).
3. Ruojin Cai, Guandao Yang, Hadar Averbuch-Elor, Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariharan. 2020. Learning Gradient Fields for Shape Generation. In Proceedings of the European Conference on Computer Vision (ECCV).
4. Frédéric Cazals and Marc Pouget. 2005. Estimating differential quantities using polynomial fitting of osculating jets. Computer Aided Geometric Design 22, 2 (2005), 121--146.
5. Yi-Ling Chen, Bing-Yu Chen, Shang-Hong Lai, and Tomoyuki Nishita. 2010. Binary orientation trees for volume and surface reconstruction from unoriented point clouds. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 2011--2019.
6. Haoqiang Fan, Hao Su, and Leonidas J Guibas. 2017. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition. 605--613.
7. Gerald B Folland. 1995. Introduction to partial differential equations. Vol. 102. Princeton university press.
8. Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 2018. A papier-mâché approach to learning 3d surface generation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 216--224.
9. Gaël Guennebaud and Markus Gross. 2007. Algebraic point set surfaces. In ACM SIGGRAPH 2007 papers. 23--es.
10. Paul Guerrero, Yanir Kleiman, Maks Ovsjanikov, and Niloy J Mitra. 2018. Pcpnet learning local shape properties from raw point clouds. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 75--85.
11. Swaminathan Gurumurthy and Shubham Agrawal. 2019. High fidelity semantic shape completion for point clouds using latent optimization. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1099--1108.
12. Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. 2019. MeshCNN: A Network with an Edge. ACM Trans. Graph. 38, 4, Article 90 (July 2019), 12 pages.
13. Rana Hanocka, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. 2020. Point2Mesh: A Self-Prior for Deformable Meshes. ACM Trans. Graph. 39, 4, Article 126 (July 2020), 12 pages.
14. Pedro Hermosilla, Tobias Ritschel, and Timo Ropinski. 2019. Total Denoising: Unsupervised learning of 3D point cloud cleaning. In Proceedings of the IEEE International Conference on Computer Vision. 52--60.
15. Amir Hertz, Rana Hanocka, Raja Giryes, and Daniel Cohen-Or. 2020. PointGMM: a Neural GMM Network for Point Clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12054--12063.
16. Hugues Hoppe, Tony DeRose, Tom Duchamp, John McDonald, and Werner Stuetzle. 1992. Surface reconstruction from unorganized points. In Proceedings of the 19th annual conference on Computer graphics and interactive techniques. 71--78.
17. Chao Huang, Ruihui Li, Xianzhi Li, and Chi-Wing Fu. 2020. Non-Local Part-Aware Point Cloud Denoising. arXiv preprint arXiv:2003.06631 (2020).
18. Hui Huang, Dan Li, Hao Zhang, Uri Ascher, and Daniel Cohen-Or. 2009. Consolidation of unorganized point clouds for surface reconstruction. ACM transactions on graphics (TOG) 28, 5 (2009), 1--7.
19. Zhiyang Huang, Nathan Carr, and Tao Ju. 2019. Variational implicit point set surfaces. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1--13.
20. Johannes Jakob, Christoph Buchenau, and Michael Guthe. 2019. Parallel globally consistent normal orientation of raw unorganized point clouds. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 163--173.
21. Li Jiang, Shaoshuai Shi, Xiaojuan Qi, and Jiaya Jia. 2018. Gal: Geometric adversarial loss for single-view 3d-object reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV). 802--816.
22. Sagi Katz, Ayellet Tal, and Ronen Basri. 2007. Direct visibility of point sets. In ACM SIGGRAPH 2007 papers. 24--es.
23. Michael Kazhdan. 2005. Reconstruction of solid models from oriented point sets. In Proceedings of the third Eurographics symposium on Geometry processing. 73--es.
24. Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. 2006. Poisson surface reconstruction. In Proceedings of the fourth Eurographics symposium on Geometry processing, Vol. 7.
25. Michael Kazhdan and Hugues Hoppe. 2013. Screened poisson surface reconstruction. ACM Transactions on Graphics (ToG) 32, 3 (2013), 1--13.
26. S. König and S. Gumhold. 2009. Consistent Propagation of Normal Orientations in Point Clouds. In VMV.
27. Itai Lang, Uriel Kotlicki, and Shai Avidan. 2020. Geometric Adversarial Attacks and Defenses on 3D Point Clouds. arXiv preprint arXiv:2012.05657 (2020).
28. Oliver Laric. 2012. Three D Scans. http://threedscans.com/info/ (2012).
29. David Levin. 2004. Mesh-independent surface interpolation. In Geometric modeling for scientific visualization. Springer, 37--49.
30. Chun-Liang Li, Manzil Zaheer, Yang Zhang, Barnabas Poczos, and Ruslan Salakhutdinov. 2018c. Point cloud gan. arXiv preprint arXiv:1810.05795 (2018).
31. Jiaxin Li, Ben M Chen, and Gim Hee Lee. 2018b. So-net: Self-organizing network for point cloud analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition. 9397--9406.
32. Ruihui Li, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2019. Pu-gan: a point cloud upsampling adversarial network. In Proceedings of the IEEE International Conference on Computer Vision. 7203--7212.
33. Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. 2018a. Pointcnn: Convolution on x-transformed points. In Advances in neural information processing systems. 820--830.
34. Minghua Liu, Lu Sheng, Sheng Yang, Jing Shao, and Shi-Min Hu. 2020. Morphing and sampling network for dense point cloud completion. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 11596--11603.
35. Xinhai Liu, Zhizhong Han, Xin Wen, Yu-Shen Liu, and Matthias Zwicker. 2019. L2g auto-encoder: Understanding point clouds by local-to-global reconstruction with hierarchical self-attention. In Proceedings of the 27th ACM International Conference on Multimedia. 989--997.
36. Shitong Luo and Wei Hu. 2020. Differentiable Manifold Reconstruction for Point Cloud Denoising. In Proceedings of the 28th ACM International Conference on Multimedia. 1330--1338.
37. Viní cius Mello, Luiz Velho, and Gabriel Taubin. 2003. Estimating the in/out function of a surface represented by points. In Proceedings of the eighth ACM symposium on Solid modeling and applications. 108--114.
38. Gal Metzer, Rana Hanocka, Raja Giryes, and Daniel Cohen-Or. 2020. Self-Sampling for Neural Point Cloud Consolidation. arXiv preprint arXiv:2008.06471 (2020).
39. Niloy J Mitra and An Nguyen. 2003. Estimating surface normals in noisy point cloud data. In Proceedings of the nineteenth annual symposium on Computational geometry. 322--328.
40. Patrick Mullen, Fernando De Goes, Mathieu Desbrun, David Cohen-Steiner, and Pierre Alliez. 2010. Signing the unsigned: Robust surface reconstruction from raw pointsets. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 1733--1741.
41. Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Love-grove. 2019. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
42. Mark Pauly, Richard Keiser, Leif P Kobbelt, and Markus Gross. 2003. Shape modeling with point-sampled geometry. ACM Transactions on Graphics (TOG) 22, 3 (2003), 641--650.
43. Francesca Pistilli, Giulia Fracastoro, Diego Valsesia, and Enrico Magli. 2020. Learning Graph-Convolutional Representations for Point Cloud Denoising. In European Conference on Computer Vision. Springer, 103--118.
44. Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017a. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 652--660.
45. Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017b. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems 30 (2017), 5099--5108.
46. Marie-Julie Rakotosaona, Vittorio La Barbera, Paul Guerrero, Niloy J Mitra, and Maks Ovsjanikov. 2020. Pointcleannet: Learning to denoise and remove outliers from dense point clouds. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 185--203.
47. Javier Romero, Dimitrios Tzionas, and Michael J. Black. 2017. Embodied Hands: Modeling and Capturing Hands and Bodies Together. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) 36, 6 (Nov. 2017).
48. Muhammad Sarmad, Hyunjoo Jenny Lee, and Young Min Kim. 2019. Rl-gan-net: A reinforcement learning agent controlled gan network for real-time point cloud shape completion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5898--5907.
49. Nico Schertler, Bogdan Savchynskyy, and Stefan Gumhold. 2017. Towards globally optimal normal orientations for large point clouds. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 197--208.
50. Lee M Seversky, Matt S Berger, and Lijun Yin. 2011. Harmonic point cloud orientation. Computers & Graphics 35, 3 (2011), 492--499.
51. Oana Sidi, Oliver van Kaick, Yanir Kleiman, Hao Zhang, and Daniel Cohen-Or. 2011. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. In Proceedings of the 2011 SIGGRAPH Asia Conference. 1--10.
52. Yongbin Sun, Yue Wang, Ziwei Liu, Joshua Siegel, and Sanjay Sarma. 2020. Pointgrow: Autoregressively learned point cloud generation with self-attention. In The IEEE Winter Conference on Applications of Computer Vision. 61--70.
53. Lyne P Tchapmi, Vineet Kosaraju, Hamid Rezatofighi, Ian Reid, and Silvio Savarese. 2019. Topnet: Structural point cloud decoder. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 383--392.
54. The CGAL Project. 2020. CGAL User and Reference Manual (5.2 ed.). CGAL Editorial Board. https://doc.cgal.org/5.2/Manual/packages.html
55. Christian Walder, Olivier Chapelle, and Bernhard Schölkopf. 2005. Implicit surface modelling as an eigenvalue problem. In Proceedings of the 22nd international conference on Machine learning. 936--939.
56. Xiaogang Wang, Marcelo H Ang Jr, and Gim Hee Lee. 2020. Point Cloud Completion by Learning Shape Priors. arXiv preprint arXiv:2008.00394 (2020).
57. Yunhai Wang, Shmulik Asafi, Oliver Van Kaick, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2012. Active co-analysis of a set of shapes. ACM Transactions on Graphics (TOG) 31, 6 (2012), 1--10.
58. Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2019. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog) 38, 5 (2019), 1--12.
59. Xin Wen, Tianyang Li, Zhizhong Han, and Yu-Shen Liu. 2020. Point cloud completion by skip-attention network with hierarchical folding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1939--1948.
60. Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. 2019. Deep geometric prior for surface reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 10130--10139.
61. Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1912--1920.
62. Hui Xie, Kevin T McDonnell, and Hong Qin. 2004. Surface reconstruction of noisy and defective data sets. In IEEE Visualization 2004. IEEE, 259--266.
63. Hui Xie, Jianning Wang, Jing Hua, Hong Qin, and Arie Kaufman. 2003. Piecewise C/sup 1/continuous surface reconstruction of noisy point clouds via local implicit quadric regression. In IEEE Visualization, 2003. VIS 2003. IEEE, 91--98.
64. Minfeng Xu, Shiqing Xin, and Changhe Tu. 2018. Towards globally optimal normal orientations for thin surfaces. Computers & Graphics 75 (2018), 36--43.
65. Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. 2019. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE International Conference on Computer Vision. 4541--4550.
66. Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. 2018. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 206--215.
67. Wang Yifan, Felice Serena, Shihao Wu, Cengiz Öztireli, and Olga Sorkine-Hornung. 2019a. Differentiable surface splatting for point-based geometry processing. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1--14.
68. Wang Yifan, Shihao Wu, Hui Huang, Daniel Cohen-Or, and Olga Sorkine-Hornung. 2019b. Patch-based progressive 3d point set upsampling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5958--5967.
69. Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2018a. Ec-net: an edge-aware point set consolidation network. In Proceedings of the European Conference on Computer Vision (ECCV). 386--402.
70. Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2018b. Pu-net: Point cloud upsampling network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2790--2799.
71. Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, and Martial Hebert. 2018. Pcn: Point completion network. In 2018 International Conference on 3D Vision (3DV). IEEE, 728--737.
72. Yongheng Zhao, Tolga Birdal, Haowen Deng, and Federico Tombari. 2019. 3D point capsule networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1009--1018.
73. Qingnan Zhou and Alec Jacobson. 2016. Thingi10K: A Dataset of 10,000 3D-Printing Models. arXiv preprint arXiv:1605.04797 (2016).

Index Terms

• Computing methodologies

  • Computer graphics

    • Shape modeling

      • Mesh models

      • Point-based models

  • Machine learning

    • Machine learning approaches

      • Neural networks