Interval Shading: using Mesh Shaders to generate shading intervals for volume rendering

We propose to use tetrahedrons as primitives for volume rendering and a pipeline to rasterize them. Our work relies on the recently introduced mesh shaders to encode each tetrahedron such that the rasterizer computes the depths of the front and back faces at the same time when interpolating vertices attributes. Then, the fragment shader receives the two depths and


INTRODUCTION
In computer graphics volume rendering is an essential tool for representing complex scenes.Volume rendering can be used to represent complex phenomena (clouds [Bouthors et al. 2008], particle effects [Cha et al. 2009], etc.) for the creation of VFX in movies or video games.It can also be used for the visualization of complex mechanical systems in CAD [Huang and Carter 2005], for scientific visualization [Anderson et al. 2007], and for representing transparent materials [Everitt 2001].
Nowadays graphic cards are optimized to render surfaces and support the rendering of points, edges, and triangles.However, despite the popularity of volume rendering, there is no hardware acceleration for the rasterization of 3D primitives.As a result, volume rendering methods do not take full advantage of the graphic card and the vast majority of these algorithms are executed in fragment shaders.This can be explained by the similarity of the rasterization for 0, 1, and 2D primitives as solving their on-screen projection create a single solution per pixel: a fragment with a depth, interpolated attributes, etc.In comparison, solving the on-screen projection of a 3D primitive creates multiple solutions for each pixel that have to be solved simultaneously, stored contiguously, and sorted according to their depth.Doing so naively would create a bottleneck in the rendering pipeline by introducing synchronization points.
However, the recent introduction of the mesh shading pipeline ( [Kubisch 2018;Moore 2023;Oberberger et al. 2023]) allowed for the use of more complex rendering primitives as long as they could be expressed as points, lines, or triangles at the end of the mesh shading stage.The mesh shading pipeline replaces the vertex, geometry, and tesselation stage with an optional task stage and mesh stage.The mesh stage resembles the compute stage and supports the same kind of inputs, but its outputs are geometric primitives that are directly fed to the rasterizer.Similar to the compute Author's address: Thibault Tricard, thibault.tricad@inria.fr,Maverick, INPG, LJK, Inria, Grenoble, France.shader, the mesh shader uses a workgroup to distribute complex operations on multiple threads that share parts of their memory.This grants more control over the geometry than the geometry and tesselation stages.
In the context of explicit surface rendering, mesh shading pipeline opened new possibilities for continuous level-of-detail approaches [Englert 2020], for on-the-fly tesselation [Santerre et al. 2020], and for early culling of geometry [Unterguggenberger et al. 2021] by using tailored meshlet generation strategies [Jensen et al. 2023].Mesh shading can also be used to decompress meshlet in real time [Kuth et al. 2024].Recently [Kreskowski et al. 2022] proposed to use mesh shaders to render implicit surfaces using rasterization rather than marching algorithms.While this greatly reduces the cost of isosurface rendering, this method cannot be used in volume rendering.However, to the best of our knowledge, no method has been proposed to process volume primitives.
In this article, we propose to use tetrahedrons as a rendering primitive.Tetrahedrons are already popular among the computer graphics community: they support being deformed [Gascon et al. 2013], and they are used in mechanical simulation [Koschier et al. 2014] and fluid simulation [Ando et al. 2013].
We propose a method that takes advantage of the new possibilities offered by the mesh shading pipeline to process tetrahedrons and use them to invoke Interval Shaders.Here, we introduce an Interval Shader as a fragment shader that receives a depth interval for a single fragment.The first value corresponds to the depth of the front faces of the tetrahedron and the second to the back faces.The interval shader uses this depth interval to compute the current fragment's color for volume rendering.As a fragment can only have one depth on current hardware, we propose a method to process tetrahedrons using mesh shaders to hijack the rasterizer to compute two depths per fragment.

METHOD
In this section, we propose a method to invoke an interval shader using a mesh shader.First, we address the generation of depth intervals in a simple case with strict constraints (Section 2.1), then we show a method to process tetrahedrons so they match these constraints 2.2.Let's consider a triangular prismoid 1 in projected space composed of six vertices   with  ∈ [0, 5], with  0 ,  1 ,  2 forming the first triangular base, and  3 ,  4 ,  5 forming the second (see Figure 2).We place ourselves in the case where both bases of the prismoid project to a single triangle in screen space such that :

Generating Depth Intervals: Simple Case
. Any point  on this triangle is defined as follows : with  0 ,  1 , and  2 being the screen space barycentric coordinates of the point p.Note that as we are using screen space barycentric coordinates we can use the same set for the first and second bases of the prism.We then find the z coordinates of  using: with . 0 the z coordinate of the point  on the first base of the prism and . 1 the z coordinate of the point  on the second.We finally obtain,   = (.0 , . 1 ) and   =  (. 0 , . 1 ).Now that we know how to compute   and   , we propose to hijack the rasterizer into computing them for us, so as to have them as input variables in the fragment stage.To do so, we emit a triangle proxy for the prismoid with the following vertices coordinates: .By setting the z coordinates to 0 we force the rasterizer to use the screen space barycentric coordinates to interpolate the vertices attributes.The interpolation of the  by the screen space barycentric coordinates is equivalent to the equation 2 and 3. Thus we have 1/  and 1/  as input of the fragment shader.We can now use this interval for our shading.

Tetrahedrons
This approach is only valid for a prismoid whose bases project onto a single triangle in screen space, or for a shape that has been decomposed into such prismoids.While a prismoid decomposition would be a complex task in a general case, in the specific case of a tetrahedron, this operation can be solved analytically in real time in a mesh shader.For a tetrahedron, there are only two cases to consider (Figure 3): • Case 1: when the tetrahedron projects onto a single triangle in screen space, the decomposition creates three triangle prismoids.• Case 2: when the tetrahedron projects onto a quad in screen space, the decomposition creates four triangle prismoids.
Specific cases where one or two faces of the tetrahedron have a null area in screen space can be handled by the first case with a numerically stable implementation.2.2.1 Emitting the proxy.As, in Section 2.1 we emitted one triangle proxy for the prismoid we processed, here we need to emit proxies composed of as many triangles as we have prismoid: 3 in case 1 and 4 in case 2. To create the proxies we must compute the position of all required points in screen space, and their depths in the projected space.We start by defining   the  ℎ vertices of the tetrahedron with  ∈ {0, 3}, and   the  ℎ vertices of the proxy.
Case 1.In this case, we define  0:2 as the points on the silhouette of the proxy and  3 as the point not on the silhouette.With:  0:3 =  : ./: .With ,,, and  the indices of the vertices on the tetrahedron, chosen such that  0:2 are ordered in a clockwise manner as shown in Figure 3.For all vertices  0:2 on the silhouette, we define: : . : .For the vertices  3 , we compute its screen space barycentric coordinates  on the triangle  0:2 using the edge function [Pineda 1988] such that: Then we project back  3 on both the front and back faces to find its depths.As  3 is the screen space projection of   we find: 0 =   ./ .To find the second projection of  3 on either the front face or back face: we follow the Equation 2: 1 Then we have   = ( 0 ,  1 ) and   =  ( 0 ,  1 ).

Clipping
As explained in Section 2.1, when creating the proxy, we store the inverse of the depths as vertex attributes.Thus, if the   value gets too close to zero (when the tetrahedron is too close to the near plane), the interpolation 1/  will create numerical instability.To avoid this effect, we propose to clip the tetrahedrons before the near plane (at   =  +  in camera space).Depending on the number of vertices having a z coordinate inferior to   this operation can create two shapes: either a prism-like shape (two triangles, three parallelograms), or a tetrahedron (see Figure 4).If the output is a prism-like shape we decompose it into three tetrahedrons and process them as described in Section 2.2.There are multiple valid ways to split this shape into tetrahedrons.Given the indexing shown in Figure 4 we split it as follows: {0, 1, 2, 3}, {1, 2, 3, 5}, {1, 3, 4, 5}.In addition to avoiding unexpected behavior, clipping the tetrahedron before the near plane allows us to place our camera inside a (or a set of) tetrahedron, as can be seen in Figure 8.In this Section, we show some results obtained with our method.Figure 5 shows the result of our method while dealing with a single tetrahedron.Figure 5 Left shows the depth of the front faces, Figure 5 Center shows the depth of the back faces, and Figure 5 Right shows the difference in depth between the front and the back faces.This computation is done in one draw call as the fragment shader receives the depth of the front and back faces.This allows us to exploit the depth intervals in two main ways :

RESULTS
• To compute the optical depth of a tetrahedral mesh as described in Section 3.1 • As bounds for a marching algorithm as described in Section 3.2

Optical Depth
Figure 6 demonstrates how our method can be used to compute the optical depth in tetrahedral meshes.To achieve this result we chose a mesh from cgtrader [The-Ni11 2018] and converted it into a tetrahedral mesh using the Geogram library [Levy 2015].The resulting mesh was directly fed as an indexed tetrahedron list to our mesh shaders without modification or reordering.Then in the fragment shader, given the difference in depth generated for each fragment, we compute the position of the front and back faces in world space, and output the norm of their differences.
Following that, we rely on the blending phase of the pipeline to sum those lengths (using the add blend function and blending parameter equal to one) which gives us the distance spent in the model for each pixel.Finally, we apply the Beer-Lambert law as a post-process to evaluate the light transmitted by the model.This allows us to evaluate the transmittance in any tetrahedron mesh in one draw call followed by a simple post-process.Our method is not limited by the number of intersections between the camera ray and the surface as opposed to A-Buffer-like approaches [Everitt 2001] and gives exact results.

Ray Marching
Our method can also be used to render signed distance fields.In Figure 7 we show an asteroid field ray-marched with a sphere tracing algorithm [Hart 1996].To achieve this we instantiate multiple tetrahedrons, each bounding a procedural signed distance field defined using Hypertextures [Perlin and Hoffert 1989] in a frame centered on the tetrahedron.
In this case, each fragment only computes the ray-SDF intersection for only one asteroid, and we rely on the depth buffer to choose which fragment to show on screen.This allows us to distribute the computation of the ray-SDF intersection on multiple fragments per ray, thus limiting the complexity of the marching done in each fragment, but preventing us from stopping the marching at the first intersection.As, in most cases, the cost of an image is determined by the most expensive fragment, splitting the computation of one ray into multiple fragments reduces the overall cost of the rendering.This is particularly useful in the case of sphere tracing [Hart 1996] where the cost of a fragment increases greatly for grazing rays.Here the worst-case scenario would be rays that closely miss multiple surfaces.Our method distributes those close misses on various fragments that can be executed in parallel depending on warp availability.

PERFORMANCES
All our examples have been rendered on an NVIDIA GeForce RTX 4080 and are rendered in 4K resolution (3 840 × 2 160) using Vulkan 1.3.242.Each mesh shading work group processes one input tetrahedron, using one thread per work group.The performances of our method are described in Table 1.Our measure shows that our method remains real-time even when used to invoke ray marching (see Section 3.2).When comparing the rendering time of the crystal to the asteroids that use a similar amount of tetrahedrons we can see that most of the time is spent in the ray marching algorithm.Our example, in Figure 8, shows that our method can handle large amounts of primitive while remaining real-time even when clipping is required.Fig. 7. Field of asteroids evaluated with our method inspired by [Neyret 2024].Top left: asteroid shaded using the number of steps of sphere marching before reaching the surface.Top right: rendering of the bounding tetrahedron shaded using the interval length to march.Bottom left: cost of the visible fragment (black means 0 sphere marching steps, white means 60 sphere marching steps).Bottom right: sum of the cost of all fragments evaluated (black means 0 sphere marching steps, white means 60 sphere marching steps).

Model
Figure

LIMITATIONS
The method we propose allows us to generate shading intervals that can be evaluated in parallel and then recombined in the blending phase.As a result, it can only be applied to render volume information that is not order-dependent.
Our method uses screen space proxies to force the rasterizer to use screen space barycentric coordinates to interpolate depths correctly.In consequence, the interpolation of vertice attributes cannot be computed correctly by the rasterizer.Hence, our methods can not be applied to render tetrahedrons with vertex attributes, only per tetrahedron information can be used for the shading.An example of this can be shown in Figure 6 Center Left where each tetrahedron has a different absorption color, and in Figure 7 where each tetrahedron has a different model matrix that is inverted in the fragment shader.
Currently, our method can only handle one tetrahedron per mesh shading work group which limits the number of tetrahedrons that can be drawn at the same time.Attempts at processing more tetrahedrons per work group resulted in lower performances either due to branch divergence in the mesh shader or to inconsistent memory access when fetching tetrahedrons.This limitation could potentially be addressed by reorganizing tetrahedrons in meshlet-like structures.

Fig. 2 .
Fig. 2. Illustration of a prismoid in projected space (left), and in world space (right).

Fig. 3 .
Fig. 3. Illustration of two possible cases of the prismoid decomposition for a tetrahedron.left: Wireframe of the tetrahedron.right: Proxy generated by the mesh shader with the vertex indices used.top: First case, three triangles and and four vertices are created.bottom: Second case, four triangles and five vertices are created.

Fig. 4 .
Fig. 4. Clipping of a tetrahedron by a plane.Top: the tetrahedron, the clipping plane, and its intersection with the tetrahedron in red, in yellow the clipped vertices.Bottom: The clipped tetrahedron and the indices we use to describe them.Left: one vertex clipped by the plane.Center: two vertices clipped by the plane.Right: three vertices clipped by the plane.

Fig. 5 .
Fig. 5. Single tetrahedron processed by our method.Left: depth of the front face, Center: depth of the back faces, Right: difference in depth in world space

Fig. 6 .
Fig. 6.Crystal rendered using our method.a: sum of interval length.b: Light transmittance where each tetrahedron has a different absorption color.b: Light transmittance where each tetrahedron absorbs the same color.d: Light transmittance and reflection based on an environment map.

Table 1 .
Performances measures of our examples