A Novel Approach for Repairing Unsegmented Liver Vascular Images based on Centerline

In liver vascular intervention surgery, the absence or narrowing vascular images may impact the understanding of patients’ vascular structure and morphology. Repairing and analyzing vascular images is a crucial task in computer-aided diagnosis and surgery for minimally invasive vascular diseases. Previous research has addressed the problem of repairing small areas with regular shapes or minor curvature changes in the holes, often leading to modifications to the existing points and encounter noise and geometric loss. Unlike existing point cloud completion algorithms, this paper aims to address the issue of missing vessels in liver vascular images that have not been segmented. It employs axis-aligned bounding boxes to extract missing vascular images. Based on the geometric characteristics of the liver vascular centerline, two methods of cross-section and spherical wave propagation are proposed for repair process. These methods take localized missing vascular images as input and preserve the complete topological structure of the vessels. The repair results are evaluated using the Chamfer distance and Dice similarity coefficient. Experimental results demonstrate that the proposed approach effectively repairs large-scale, non-closed missing vascular images in liver vascular images and generates complete vessel geometric models with accurate liver vascular structure and morphology. The validity of the vessel models is confirmed through hemodynamic analysis, providing robust support for medical image analysis and diagnosis.


INTRODUCTION
Liver vascular diseases, such as vascular malformations, liver artery aneurysms, and liver venous thrombosis, can affect the structure and function of the liver's blood vessels [12].Interventional surgery is a common treatment method for liver vascular diseases.With the advancement of interventional radiology, transcatheter arterial chemoembolization (TACE) has emerged as the preferred nonsurgical treatment for liver cancer.TACE is performed by catheterization through the femoral artery, followed by selective insertion into the liver artery supplying the tumor.Through embolization and perfusion of chemotherapeutic agents, tumor-feeding vessels are primarily targeted to impede tumor blood supply, inducing ischemic necrosis within the tumor and administering chemotherapy to kill cancer cells.To guide the surgery and ensure its safety and accuracy, the patient's vascular images must be processed.Medical professionals employ various vascular imaging techniques, such as digital subtraction angiography (DSA), computed tomographic angiography (CTA), intravascular ultrasound (IVUS), optical coherence tomography (OCT), magnetic resonance imaging (MRI), and magnetic resonance angiography (MRA), to acquire and analyze the patient's blood vessels images [6].Medical image processing is an interdisciplinary field that involves computer science, medicine, and imaging.It aims to develop and apply computer technology and algorithms to acquire, analyze, interpret, and improve medical images [14].Vascular image processing is an important branch of medical image processing.Various image processing techniques, including image reconstruction, enhancement, and segmentation, are employed to derive critical information such as treatment regions, vascular pathways, and lesion severity.This facilitates the creation of personalized surgical plans.
The liver vasculature exhibits highly complex and variable morphology, making vascular image processing and analysis challenging.Due to the high complexity of the vascular structure, traditional 3D modeling methods have difficulty effectively capturing its true form.Therefore, researchers have begun to use centerline-based methods to describe vascular topological structures.The centerline refers to the trajectory of the center of the maximum inscribed sphere inside an object [17].This representation method has many advantages, including the ability to capture branching and connection relationships of the vascular structure, the elimination of interference factors such as noise and local deformation, and compatibility with other computer vision algorithms.The centerline-based reconstruction process [16], techniques such as image processing and mathematical modeling are generally used to extract vascular centerline information to create a complete 3D vascular geometric model.Compared with traditional methods, centerline-based reconstruction technology can more accurately reconstruct vascular structures, and has important applications in medical imaging diagnosis, disease prediction, and treatment planning.
This study is a subtask of the real-time liver vascular interventional navigation and warning system project, which aims to explore the feasibility of using computer-aided diagnosis (CAD) technology to reduce physician workload, improve medical efficiency and accuracy, and provide vascular geometric models and analysis tools for medical research.The vascular geometric model is a 3D model generated based on imaging data, which can provide a clear and realistic surgical operation scene [18].During the surgery, the vascular geometric model is used to help doctors locate and operate, reducing radiation exposure to patients and physicians.In this study, in combination with the real-time liver vascular interventional surgery requirements, CAD technology was improved and optimized to assist doctors in selecting the best surgical route and operation method.Beyond its surgical applications, this study offers novel technical approaches and theoretical foundations for medical practitioners and researchers in related domains, thus advancing the digitalization of medical imaging diagnostic technology.
The navigation and warning system is based on scanned vascular images and selects a guide wire entry point.By employing an automatic skeletonization algorithm, the system obtains the vascular centerline and performs secondary skeletonization for vascular segmentation.Using the generated point cloud from segmentation, a localized implicit reconstruction and fitting of complex liver vascular structures can be realized, allowing real-time acquisition of liver vascular models with topological structures.The traditional computational fluid dynamics (CFD) method is employed to analyze the model [20].The liver vascular model is divided into inlet and outlet surfaces.With CFD, blood flow velocity, pressure, wall shear stress, and fluctuation characteristics can be calculated.The obtained hemodynamic parameters serve as a foundation for solute transport modeling in blood and provide necessary technical support for the implementation of automated navigational interventions.In cases where there are missing or non-segmentable regions in liver vascular images resulting in incomplete point cloud data, the vascular reconstruction and subsequent CFD analysis may be compromised.To address this issue, a repair procedure is necessary to generate a complex liver vascular model that meets the requirements for subsequent CFD analysis.
The primary objective of this study is to achieve automatic repair of blood vessels in regions with poor quality liver vascular images, ensuring that the reconstructed vascular model is suitable for subsequent blood flow analysis and assists medical professionals in analyzing liver vascular diseases.Therefore, traditional repair methods are employed to efficiently accomplish the liver vascular repair task and fulfill the requirements for subsequent blood flow analysis.
The remaining sections of this manuscript are structured as follows: Section 2 introduces the principles and related work; Section 3 describes the specific methods used in this study; Section 4 presents the evaluation indicators used in this study and compares the vascular repair methods through experiments; Section 5 provides a summary and discusses the advantages, challenges, and future research directions of the methods presented in this article for vascular image applications.

RELATED WORK
Vascular image repair is an important research direction in the field of medical imaging, aiming to generate high-quality vascular point-cloud data from blurry, low-quality vascular (see Fig. 1).In vascular imaging, various problems such as missing or broken data may occur due to noise, motion artifacts, and other factors, which can affect a physician's diagnosis and treatment decisions [2].The main goal of vascular image repair research is to use computer vision and other technical means to repair and reconstruct vascular images while maintaining the integrity of the vascular structure.
In vascular image repair, a combination of point-cloud holefilling techniques is utilized to fill in holes and gaps, creating a complete vascular structure image.Point cloud hole-filling is one of the core issues in 3D reconstruction and computer vision, as pointcloud data obtained from sensors or scanners often contains missing data and noise [5].To address this issue, numerous point-cloud hole-filling techniques have been proposed by the academic and industrial community, aimed at restoring or filling data in missing areas, resulting in more complete and accurate point cloud models.At present, there is a lack of dedicated algorithms specifically designed for repairing blood vessel images.The existing hole repair algorithms primarily focus on mending shapes and topologically simple objects.
Early techniques for point cloud hole-filling utilized algorithms based on geometric information and mathematical interpolation, Fig. 2. The overview of this Study such as interpolation using local neighborhood information and optimization combined with prior knowledge [4].Alternatively, methods involving spline interpolation were used to fill holes in accordance with constraints.The research and development of these techniques have helped to improve the quality and application of point-cloud data.However, these methods often struggle to handle complex point-cloud topologies and missing shapes, which may result in issues such as non-uniformity and unnaturalness of the results.
With advances in research, more efficient and accurate techniques for point cloud hole-filling have emerged.One of the most commonly used methods is based on 3D network models.This approach converts point-cloud data into mesh and utilizes mesh processing algorithms to repair small holes.Wen et al. [22] proposed a new triangle mesh repair algorithm, which generates a new triangle mesh and merges it with an existing 3D model to fill holes.However, handling large holes with this method can result in mesh deformation and surface roughness.Xu et al. [19] also introduced a hole-filling algorithm based on a latitude-longitude grid, which transforms the 3D coordinates of point-cloud data into spherical coordinates.This method analyzes and processes boundary curves or surfaces to obtain constraints for repairing the missing regions [9].The resulting complete point cloud has fine details, but requires abundant memory and computational resources.
In recent years, deep learning-based point cloud hole repair algorithms have received significant attention and research.Existing methods have been trained on regular object datasets.Li et al. [8] proposed a novel point cloud upsampling network based on GAN models.This method combines upsampling and point-cloud data correction, integrating features into self-attention units.The method repairs hole at the facet level, thereby limiting its effectiveness in repairing large gaps.Wang et al. [15] proposed a method for automatic sparse point cloud hole-filling using a back propagation neural network optimized by a genetic algorithm.By combining hole identification, interpolation of hole regions, and hole repair, this method automatically repairs incomplete point-cloud models, but may not perform well when repairing large gaps with a high amount of missing point-cloud data.Other point cloud hole-filling algorithms also exist.For example, surface-fitting algorithms are used to fit the holes and missing regions in point clouds to obtain shape and surface information [10].The Moving Least Squares (MLS) algorithm is used to fill gaps by fitting the surfaces of the nearest neighbor point cloud, handling irregularly shaped gaps [13].However, this method may fail when repairing point-cloud models with sudden changes in curvature.The Variational Level Set algorithm is also used for repairing complex 3D model holes.The algorithm constructs a signed distance function to represent the surface of the hole at the zero level set [21].The global convex optimization energy model is introduced using the variational level set on an implicit surface.The point-cloud data represents the global energy function, and the hole-filling is completed through voxel diffusion using a combination of convolution and synthesis in alternating steps.This algorithm effectively restores the detailed features of repaired holes and can handle large holes, but sensitive to parameter selection.
Emerging technologies and algorithms have provided more choices and solutions for point cloud hole repair.However, limited attention has been given to the study of missing liver vascular images.This may be attributed to several factors: the complex and unpredictable topology of liver blood vessels, a scarcity of liver blood vessel image datasets, and challenges in data acquisition.This paper aims to address the aforementioned issues by proposing a liver-specific repair method that utilizes the original liver blood vessel images as input, building upon relevant research in the field.

METHOD
The two methods proposed in this paper aim to effectively repair liver vascular structures that were unsegmented in the original vascular images.The proposed methods utilize constraints such as geometric features and branching information of the liver vasculature to guide the hole-filling process.High-quality vascular point-cloud data is computed, and a smooth vascular model is generated to complete the liver vascular structure repair process.For an overview of the paper's methodology, please refer to Fig. 2.

The centerline-based cross-section method
Liver vasculature can exhibit diverse variations in morphology and structure depending on the disease state.This study aims to repair local liver vascular imaging data.Based on the eccentricity of the  cross-section of the liver artery, which refers to the unequal lengths of the major and minor axes, the vascular cross-section is assumed to be elliptical.When dealing with missing vascular segments in original medical images, given two endpoint coordinates  1 = ( 1 ,  1 ,  1 ) and  2 = ( 2 ,  2 ,  2 ) of the missing segment, the center coordinate  = (  ,   ,   ) between the two endpoints is computed.An Axis-Aligned Bounding Box (AABB) is a rectangular prism aligned with the coordinate axis, containing the maximum and minimum boundaries of the target object [3].The AABB structure is simple, easy to implement, and computationally efficient.Given the center point  and  = (  ,   ,   ) representing the vascular diameter in the , , and  directions at point , the minimum coordinate is (  −   ,   −   ,   −   ), and the maximum coordinate is (  +   ,   +   ,   +   ), with respect to the world coordinate system centered at (  ,   ,   ).This method is called the center-radius representation, and the AABB representation is shown in Eq. (1).
Where   ,   , and   represent the central coordinates of the vessel in the coordinate space, while   ,   , and   represent the diameter of the vessel in the coordinate space.The vessel has vertices at  0 ,  1 , . . .,  7 , which are depicted in Fig. 3.The original vascular image is cut using an AABB bounding box, and a cubic grid is constructed to approximate the vessel.Firstly, the vascular image is subjected to a cut operation to isolate the requited area.Subsequently, the centerline of the liver vascular system (see Fig. 4(b)) is identified.Then, the set of points along this centerline is resampled to produce a new sampled point set .The local Frenet frame    is computed at each point  (), where  , , and  vectors are defined along the curve, forming a local orthogonal coordinate system at each point along the curve in 3D space [7].The Frenet formulas describe the relationship between the tangent, normal, and binormal vectors at a point on the centerline, and when  () ′ is not equal to 0,  ,  ,  are defined by Eq. ( 2) Where  () is the parameter equation of the centerline,  is the unit tangent vector, pointing in the direction of particle motion;  is the unit normal vector, obtained by differentiating  with respect to arc length and normalizing, and  is the unit binormal vector, obtained by taking the cross product of  and  .The Frenet frame can be easily computed at each point on the centerline.As shown in Fig. 4(a), the black curve represents the centerline, and the red, green, and blue arrows represent the tangent, normal, and binormal vectors at a point on the centerline, respectively.For points on the curve where the curvature is zero, the adjacent tangent, normal, and binormal vectors are used instead.
Suppose  is a point in 3D space R 3 .For sampled point  (), the coordinates of  can be converted from world coordinates to local pixel coordinates.The following Eq. 3 is used to transform  (, , ) into  (, , ).
Where , , and  are respectively the transformation matrix, origin position, and actual distance between each pixel of the original image.
With the sampling point  () as the center, and the vessel radius  at that point as the step size, the point-cloud data was obtained at intervals of  based on the  −  plane using Eq. 4. The crosssectional point cloud  was then outputted based on the recorded point-cloud data, as shown in Fig. 5(a).
Where  is a weight that controls the size of the ellipse based on the vessel curvature.Suppose  is a point in 3D space R 3 .For the generated point cloud , the coordinates of  are converted from local coordinate system to world coordinate system.The following Eq.( 5) is used to transform  (, , ) into  (, , ).
The appropriate cross-sectional point-cloud data is selected and processed based on the geometric characteristics of the vascular point cloud.Specifically, by specifying the minimum distance between two points, merging duplicate points, deleting unused points, and down-sampling the point cloud, as shown in Fig. 5(b), the final selected cross-section point-cloud data is obtained.These crosssectional point-cloud data are saved and fitted to a surface, such that the fitted surface can contain sufficient point-cloud data and better reflect the morphological characteristics of the original vascular model.

The centerline-based spherical wave propagation method
Spherical waves are a type of wave phenomena that propagate through isotropic media in a spherical manner, i.e., they spread along a surface with a spherical geometry.In contrast to other types of waves, spherical waves possess distinctive features during their propagation: their amplitude and phase remain constant at any position.Due to their isotropic nature, spherical waves propagate at the same speed in all directions.Spherical waves can accurately describe the propagation characteristics of waves emitted in all directions from a source point.Using spherical waves, the wave equation can be expressed in polar coordinates, simplifying computation and analysis.Its mathematical expression is presented in Eq. ( 6): 1 where  is the wave function,  is the distance to the surface of the sphere,  is the wave propagation velocity in the medium, and  is time.This equation describes the temporal and spatial changes of the wave function.
To store the generated point-cloud data, a triangular mesh is defined based on the size of the cut AABB image.The point set on the central line is downsampled to obtain the sampling point set .Using each sampling point  () as the center and the blood vessel radius  as the step size, a spherical wave is propagated along the central line.For each source point  (), all points that are Euclidean distance  away from the triangular mesh points are retained.The mathematical expression for the spherical wave function is used to describe the wave propagating in all directions from a source point  () and is shown in Eq. (7).
Where  is the distance to the surface of the sphere,  and  are the polar and azimuthal angles in the spherical coordinate system,  is the wave number,  is the angular frequency, and  is the wave function.An isotropic spherical wave with the opposite phase is employed to propagate, and the grid points are retained for each propagation.The grid points are uniformly sampled and converted into point-cloud data, laying the foundation for subsequent comparative experiments.

ANALYSIS OF EXPERIMENTAL RESULTS
The experimental platform utilized an Intel Core i7 -11800H @ 2.30GHz CPU with a 16GB memory and ran on a 64-bit operating system, Windows 11.The data used in this study consists of nine portal venous phase CT scan sets from patients with various primary and metastatic liver tumors.The corresponding regions of interest are blood vessels and tumors within the liver.These data were obtained from the Interventional Surgery Department of the Qingdao Municipal Hospital in Shandong Province, China.The collection of CT imaging data involved the injection of contrast agents into the blood vessels, followed by non-invasive medical imaging techniques.Ethical review was conducted prior to the experiment on liver blood vessel image repair using this dataset.The proposed method in this paper is based on the geometric features of the centerline.It applies the cross-sectional method and the spherical wave propagation method to repair the original liver blood vessel images, aiming to generate point cloud data that closely resembles real data.

Evaluation indicators
To quantitatively measure the quality of blood vessel repair, two evaluation metrics, the Chamfer Distance (CD) and Dice Similarity Coefficient (DSC), were used to compare the point-cloud data and vascular geometric model.The CD in 3D space is a method used to measure the distance between point clouds and is primarily employed in point cloud reconstruction tasks.It quantifies the similarity or difference between point clouds, and classifies or sorts them based on the calculated distance.Its mathematical definition is the average Euclidean distance between points in one point cloud and their closest point in another point cloud.The mathematical expression is shown in Eq. (8).
Where  and  represent the ground true and repaired 3D point clouds respectively.The first term represents the sum of the minimum distances between any point  in the ground true point cloud  and the repaired point cloud .The second term represents the sum of the minimum distances between any point  in the repaired point cloud  and the ground true point cloud .If these two distance metrics are large, it indicates that there is a significant difference between the ground true and repaired point clouds.Conversely, if these distances are small, it indicates a good repair effect.
The DSC is a statistical measure of the similarity between two sets.Its mathematical definition is twice the size of the intersection of the sets divided by the sum of the sizes of the sets.Specifically, the mathematical expression for the DSC is shown in Eq. (9).Where  and  represent two sets, while  refers to the total number of pixels.The DSC ranges from 0 to 1, with larger values indicating greater similarity between the two sets.A value of 0 means the sets have no intersection, while a value of 1 means the sets are identical.In general, these two methods can help to better understand the similarities and differences between different sets and evaluate their accuracy and reliability.Based on this foundation, research can be conducted to investigate the impact of different parameter settings on the repair results and explore ways to optimize the algorithm to improve its precision and robustness.

Comparison experiment
In this paper, a comparative experiment will be conducted to evaluate the two proposed algorithms.For data acquisition, a total of 9 sets of original liver blood vessel images were collected.The blood vessel images were segmented using bounding boxes, and segments with varying curvatures were extracted for comparison purposes.The endpoints  1 and  2 of the missing vessel were located and calculate the Euclidean distance between these two points as well as the center point .The missing vessel segment was cut from the original vessel image using an AABB bounding box (see Fig. 7), and local liver vessel segments were preprocessed with adaptive threshold segmentation and pixel value normalization.The interval for resampling was determined based on the curvature information from the existing centerline, where dense sampling points were taken for larger curvatures, while the gap between sampling points was set according to the vessel diameter for smaller curvatures.Point cloud data was generated using the cross-section method and the spherical wave propagation method, respectively, with the threshold and vessel diameter information at each resampled point.Local point-cloud data was obtained and transformed from local coordinates to world coordinates.
The point-cloud data was compared using the CD in this study.The CD is shown in Table 1.The repaired point-cloud data was resampled, and the resampled point-cloud data was denoted as the repaired point clouds.The original liver vessel image was subjected to level sets segmentation to obtain the vessel segmentation result [1].Then, the Marching Cubes algorithm was used to extract the vessel surface, and the point cloud extraction algorithm was eventually employed to extract the surface point-cloud data, which was represented as the ground truth.The similarity between the two sets of point-cloud data was analyzed by comparing the size of the distance metric between them.
The vascular geometric model was compared using the DSC in this study.The results of DSC measurement are presented in Table 2.The vascular geometric model was generated by using the implicit surface reconstruction algorithm based on radial basis functions (RBF) for the surface point-cloud data and the repaired point clouds mentioned earlier [11].The vascular geometric model was binarized, and the resulting binary image was denoted as the ground truth and the repaired value, respectively.The similarity between the two vascular geometric models was measured by comparing the positions and counts of pixels marked as 1 in the two binary images.In this study, we aimed to further obtain the missing parts of liver blood vessel images with various curvatures and compare the effectiveness of different methods in addressing this issue.The experimental results show that the cross-section method based on the centerline is more suitable for vascular repair tasks, as it improves the accuracy and robustness of vessel repair and is closer to real blood vessels.The detailed results of the vessel repair are presented in Fig. 6, which includes point cloud data and vessel models with different curvatures.The point cloud data generated by the crosssectional method based on the centerline is closer to the ground truth.On the other hand, the point cloud generated by the spherical wave propagation method is more uniformly distributed compared to the cross-sectional method.The density of the point cloud can be controlled by adjusting the weights.Regarding experimental errors, since our repair method takes local vessel images as input and generates missing vessel models as output, it does not affect the vascular topology in other regions.The proposed method in this study enhances the continuity of vascular repair, enabling better handling of vascular deformations and branching in the context of vascular repair tasks.Hence, it is confirmed that the proposed centerline-based blood vessel image repair algorithm in this study is effective for vascular repair tasks and maintains good vascular boundary preservation.

CONCLUSION
This study presents a novel approach to reconstruct missing blood vessels in liver vascular images by utilizing the centerline of liver vascular CT images.The technique utilizes the unsegmented vascular image data from the original images and employs an AABB bounding box to extract the local vessel segments.The proposed methods are evaluated and repaired based on two metrics: Chamfer distance and Dice similarity coefficient.Experimental results demonstrate that the generated vascular geometric models closely resemble the actual vessels, producing satisfactory results in reconstructing the vascular images.These findings provide comprehensive vascular structure information that can be utilized for vascular geometric analysis and support decision-making in the diagnosis and treatment of liver vascular diseases.However, limitations exist in terms of speed despite the improvement in repair accuracy.Therefore, future research in this field should focus on leveraging parallel optimization and implicit reconstruction techniques to enhance the speed of liver vessel repair while maintaining the desired repair outcomes.Further investigation of this approach holds promise for establishing a foundation in evaluating liver vascular stenosis and improving the accuracy of liver vascular image reconstruction.This can significantly impact CAD of liver vascular diseases.

Fig. 1 .
Fig. 1.Low-quality vascular imaging: a variety of factors such as motion artifacts, poor contrast, noise, or low resolution.

Fig. 3 .
Fig. 3.The diagram illustrates the AABB bounding box, where  0 represents the minimum point and  7 represents the maximum point.

Fig. 4 .
Fig. 4. (a)The Frenet frame at a sample point on a parametric curve.(b)The Local Centerline Extraction of Liver Blood Vessels.

Fig. 6 .
Fig. 6.Comparison experiment results.In(a)(c)(e), showing point-cloud data.In(b)(d)(f), showing vascular geometric models.From top to bottom: Origin list shows the missing vascular data.Cross-section list shows the vascular data repaired using a centerline-based cross-section method.Spherical wave list shows the vascular data repaired using a centerline-based spherical wave propagation method.Ground truth list shows the true vascular data.