Warp-and-project tomography for rapidly deforming objects

Abstract

Computed tomography has emerged as the method of choice for scanning complex shapes as well as interior structures of stationary objects. Recent progress has also allowed the use of CT for analyzing deforming objects and dynamic phenomena, although the deformations have been constrained to be either slow or periodic motions.

In this work we improve the tomographic reconstruction of time-varying geometries undergoing faster, non-periodic deformations. Our method uses a warp-and-project approach that allows us to introduce an essentially continuous time axis where consistency of the reconstructed shape with the projection images is enforced for the specific time and deformation state at which the image was captured. The method uses an efficient, time-adaptive solver that yields both the moving geometry as well as the deformation field.

We validate our method with extensive experiments using both synthetic and real data from a range of different application scenarios.

References

1. Anders H Andersen and Avinash C Kak. 1984. Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm. Ultrasonic Imaging 6, 1 (1984), 81--94.
2. Rushil Anirudh, Hyojin Kim, Jayaraman J Thiagarajan, K Aditya Mohan, Kyle Champley, and Timo Bremer. 2018. Lose the views: Limited angle CT reconstruction via implicit sinogram completion. In Proc. CVPR, Vol. 2.
3. Bradley Atcheson, Ivo Ihrke, Wolfgang Heidrich, Art Tevs, Derek Bradley, Marcus Magnor, and Hans-Peter Seidel. 2008. Time-resolved 3D capture of non-stationary gas flows. ACM Trans. Graph. 27, 5 (2008), 132.
4. Antonin Chambolle and Thomas Pock. 2011. A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging and Vision 40, 1 (2011), 120--145.
5. Guang-Hong Chen, Pascal Thériault-Lauzier, Jie Tang, Brian Nett, Shuai Leng, Joseph Zambelli, Zhihua Qi, Nicholas Bevins, Amish Raval, Scott Reeder, et al. 2012. Time-resolved interventional cardiac C-arm cone-beam CT: An application of the PICCS algorithm. IEEE Trans. Med. Img. 31, 4 (2012), 907--923.
6. Thomas De Schryver, Manuel Dierick, Marjolein Heyndrickx, Jeroen Van Stappen, Marijn A Boone, Luc Van Hoorebeke, and Matthieu N Boone. 2018. Motion compensated micro-CT reconstruction for in-situ analysis of dynamic processes. Scientific reports 8, 1 (2018), 7655.
7. Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip Davidson, Sean Ryan Fanello, Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, et al. 2016. Fusion4D: Real-time performance capture of challenging scenes. ACM Trans. Graph. 35, 4 (2016), 114.
8. Gerrit E Elsinga, Fulvio Scarano, Bernhard Wieneke, and Bas W van Oudheusden. 2006. Tomographic particle image velocimetry. Exp. Fluids 41, 6 (2006), 933--947.
9. LA Feldkamp, LC Davis, and JW Kress. 1984. Practical cone-beam algorithm. JOSA A 1, 6 (1984), 612--619.
10. Pascal Getreuer. 2012. Rudin-Osher-Fatemi total variation denoising using split Bregman. Image Processing On Line 2 (2012), 74--95.
11. Richard Gordon, Robert Bender, and Gabor T Herman. 1970. Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography. Journal of theoretical Biology 29, 3 (1970), 471--481.
12. James Gregson, Ivo Ihrke, Nils Thuerey, and Wolfgang Heidrich. 2014. From capture to simulation: connecting forward and inverse problems in fluids. ACM Trans. Graph. 33, 4 (2014), 139.
13. James Gregson, Michael Krimerman, Matthias B Hullin, and Wolfgang Heidrich. 2012. Stochastic tomography and its applications in 3D imaging of mixing fluids. ACM Trans. Graph. 31, 4 (2012), 52--1.
14. Samuel W Hasinoff and Kiriakos N Kutulakos. 2007. Photo-consistent reconstruction of semitransparent scenes by density-sheet decomposition. IEEE Trans. PAMI 29, 5 (2007), 870--885.
15. François Hild, Hugo Leclerc, and Stéphane Roux. 2014. Performing DVC at the Voxel Scale. In Advancement of Optical Methods in Experimental Mechanics, Volume 3. 209--215.
16. Berthold KP Horn and Brian G Schunck. 1981. Determining optical flow. Artificial Intelligence 17, 1--3 (1981), 185--203.
17. Peter J Huber. 2011. Robust statistics. In Int. Encyclopedia of Statistical Science. 1248--1251.
18. Ivo Ihrke and Marcus Magnor. 2004. Image-based tomographic reconstruction of flames. In Proc. SCA. 365--373.
19. Takashi Ijiri, Shin Yoshizawa, Hideo Yokota, and Takeo Igarashi. 2014. Flower modeling via X-ray computed tomography. ACM Trans. Graph. 33, 4 (2014), 48.
20. Matthias Innmann, Michael Zollhöfer, Matthias Nießner, Christian Theobalt, and Marc Stamminger. 2016. VolumeDeform: Real-time volumetric non-rigid reconstruction. In Proc. ECCV. Springer, 362--379.
21. Clément Jailin and Stéphane Roux. 2018. Dynamic Tomographic Reconstruction of Deforming Volumes. Materials 11, 8 (2018), 1395.
22. Joël Lachambre, Julien Réthoré, Arnaud Weck, and Jean-Yves Buffiere. 2015. Extraction of stress intensity factors for 3D small fatigue cracks using digital volume correlation and X-ray tomography. Int. J. Fatigue 71 (2015), 3--10.
23. Hugo Leclerc, Stéphane Roux, and François Hild. 2015. Projection savings in CT-based digital volume correlation. Exp. Mech. 55, 1 (2015), 275--287.
24. Tianfang Li, Albert Koong, and Lei Xing. 2007. Enhanced 4D cone-beam CT with inter-phase motion model. Medical physics 34, 9 (2007), 3688--3695.
25. Yangyan Li, Xiaochen Fan, Niloy J Mitra, Daniel Chamovitz, Daniel Cohen-Or, and Baoquan Chen. 2013. Analyzing growing plants from 4D point cloud data. ACM Trans. Graph. 32, 6 (2013), 157.
26. Enric Meinhardt-Llopis, Javier Sánchez Pérez, and Daniel Kondermann. 2013. Horn-Schunck Optical Flow with a Multi-Scale Strategy. Image Processing On Line, 2013:151--172, 2013. (2013).
27. Thilo F Morgeneyer, Lukas Helfen, Hazem Mubarak, and François Hild. 2013. 3D digital volume correlation of synchrotron radiation laminography images of ductile crack initiation: an initial feasibility study. Exp. Mech. 53, 4 (2013), 543--556.
28. Cyril Mory, Vincent Auvray, Bo Zhang, Michael Grass, Dirk Schäfer, S James Chen, John D Carroll, Simon Rit, Françoise Peyrin, Philippe Douek, et al. 2014. Cardiac C-arm computed tomography using a 3D + time ROI reconstruction method with spatial and temporal regularization. Med. Phys. 41, 2 (2014).
29. J Neggers, JPM Hoefnagels, MGD Geers, F Hild, and S Roux. 2015. Time-resolved integrated digital image correlation. Internat. J. Numer. Methods Engrg. 103, 3 (2015), 157--182.
30. Neal Parikh, Stephen Boyd, et al. 2014. Proximal algorithms. Foundations and Trends® in Optimization 1, 3 (2014), 127--239.
31. Alex Reche-Martinez, Ignacio Martin, and George Drettakis. 2004. Volumetric reconstruction and interactive rendering of trees from photographs. ACM Trans. Graph. 23, 3 (2004), 720--727.
32. Mai L Schmidt, Per R Poulsen, Jakob Toftegaard, Lone Hoffmann, David Hansen, and Thomas S Sørensen. 2014. Clinical use of iterative 4D-cone beam computed tomography reconstructions to investigate respiratory tumor motion in lung cancer patients. Acta Oncologica 53, 8 (2014), 1107--1113.
33. SM Shah, F Gray, JP Crawshaw, and ES Boek. 2016. Micro-computed tomography pore-scale study of flow in porous media: Effect of voxel resolution. Advances in water resources 95 (2016), 276--287.
34. SM Shah, J Yang, John P Crawshaw, O Gharbi, Edo S Boek, et al. 2013. Predicting porosity and permeability of carbonate rocks from core-scale to pore-scale using medical CT, confocal laser scanning microscopy and micro CT. In SPE Annual Technical Conference and Exhibition. Society of Petroleum Engineers.
35. Abhishek Shastry, Paolo Palacio-Mancheno, Karl Braeckman, Sander Vanheule, Ivan Josipovic, Frederic Van Assche, Eric Robles, Veerle Cnudde, Luc Van Hoorebeke, and Matthieu Boone. 2018. In-Situ High Resolution Dynamic X-ray Microtomographic Imaging of Olive Oil Removal in Kitchen Sponges by Squeezing and Rinsing. Materials 11, 8 (2018), 1482.
36. Jan-Jakob Sonke, Lambert Zijp, Peter Remeijer, and Marcel van Herk. 2005. Respiratory correlated cone beam CT. Med. Phys. 32, 4 (2005), 1176--1186.
37. Wolfgang H Stuppy, Jessica A Maisano, Matthew W Colbert, Paula J Rudall, and Timothy B Rowe. 2003. Three-dimensional analysis of plant structure using high-resolution X-ray computed tomography. Trends in Plant Science 8, 1 (2003), 2--6.
38. Thibault Taillandier-Thomas, Stéphane Roux, and François Hild. 2016. Soft route to 4D tomography. Phys. Rev. Letters 117, 2 (2016), 025501.
39. Oliver Taubmann, Günter Lauritsch, Andreas Maier, Rebecca Fahrig, and Joachim Hornegger. 2015. Estimate, compensate, iterate: joint motion estimation and compensation in 4-D cardiac C-arm computed tomography. In Proc. Int. Conf. on Medical Image Computing and Computer-Assisted Intervention. 579--586.
40. Nils Thuerey and Tobias Pfaff. 2018. MantaFlow. (2018). http://mantaflow.com.
41. Borislav Trifonov, Derek Bradley, and Wolfgang Heidrich. 2006. Tomographic reconstruction of transparent objects. In Proc. EGSR.
42. Huamin Wang, Miao Liao, Qing Zhang, Ruigang Yang, and Greg Turk. 2009. Physically guided liquid surface modeling from videos. ACM Trans. Graph. 28, 3 (2009), 90.
43. Sen Wang, Xinxin Zuo, Chao Du, Runxiao Wang, Jiangbin Zheng, and Ruigang Yang. 2018. Dynamic Non-Rigid Objects Reconstruction with a Single RGB-D Sensor. Sensors 18, 3 (2018), 886.
44. O. Weißenborn, S. Geller, M. Gude, F. Post, S. Praetorius, A. Voigt, and S. Aland. 2016. Deformation Analysis of Polymer Foams under Compression Load using in situ computed Tomography and Finite Element Simulation Methods. In 17th European conference on Composite Materials (ECCM-17).
45. Guangming Zang, Mohamed Aly, Ramzi Idoughi, Peter Wonka, and Wolfgang Heidrich. 2018a. Super-Resolution and Sparse View CT Reconstruction. (2018).
46. Guangming Zang, Ramzi Idoughi, Ran Tao, Gilles Lubineau, Peter Wonka, and Wolfgang Heidrich. 2018b. Space-time Tomography for Continuously Deforming Objects. ACM Trans. Graph. 37, 4 (2018), 36.
47. Rongping Zeng, Jeffrey A Fessler, and James M Balter. 2007. Estimating 3-D respiratory motion from orbiting views by tomographic image registration. IEEE Trans. Med. Img. 26, 2 (2007), 153--163.
48. Shuang Zhao, Wenzel Jakob, Steve Marschner, and Kavita Bala. 2011. Building volumetric appearance models of fabric using micro CT imaging. ACM Trans. Graph. 30, 4 (2011), 44.
49. Qian Zheng, Xiaochen Fan, Minglun Gong, Andrei Sharf, Oliver Deussen, and Hui Huang. 2017. 4D Reconstruction of Blooming Flowers. CGF 36, 6 (2017), 405--417.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

        • 3D imaging

        • Motion capture

  • Computer graphics