Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth

Various classic reasoning problems with natural hypergraph representations are known to be tractable if a hypertree decomposition (HD) of low width exists. The resulting algorithms are attractive for practical use in fields like databases and constraint satisfaction. However, algorithmic use of HDs relies on the difficult task of first computing a decomposition of the hypergraph underlying a given problem instance, which is then used to guide the algorithm for this particular instance. The performance of purely sequential methods for computing HDs is inherently limited, yet the problem is, theoretically, amenable to parallelisation. In this article, we propose the first algorithm for computing hypertree decompositions that is well suited for parallelisation. The newly proposed algorithm log-k-decomp requires only a logarithmic number of recursion levels and additionally allows for highly parallelised pruning of the search space by restriction to so-called balanced separators. We provide a detailed experimental evaluation over the HyperBench benchmark and demonstrate that log-k-decomp outperforms the current state of the art significantly.


INTRODUCTION
Hypertree decompositions (HDs) [24] have been demonstrated to be a valuable tool in a wide field of algorithmic applications.By way of structural decomposition of the hypergraph representation of problem instances, they induce tractable fragments for fundamental reasoning problems such as conjunctive query evaluation [24], constraint satisfaction problems [21], and related counting problems [29].Other applications can be found in game theory, where problems such as determining Nash Equilibria [16] and combinatorial auctions [15] also become tractable in cases where HDs of bounded width exist.
In many of the listed cases, we do not only have theoretical tractability results but in fact know of algorithms that are suitable for practical applications.For example, in conjunctive query evaluation, HDs can be used for efficient reduction to an acyclic instance, which allows for lineartime solving using Yannakakis' algorithm [33].Beyond practical algorithms, many of the listed problems are in fact known to be contained in the complexity class NC 2 [8] if a bounded width HD exists [3,22,23].Importantly, problems in NC 2 are considered to be highly parallelisable [8] and thus the use of HDs in these areas can be even more attractive in parallelised and distributed scenarios.The promising theoretical properties of hypertree decompositions have also been experimentally verified.Implementations in specialised database systems have demonstrated the applicability of HDs in query evaluation by using them (and closely related variants), especially on difficult instances where current heuristic-based systems struggle [1,13,14].
Despite these desirable properties and a demand for worst-case guarantees in various potential fields of application, the adoption of hypertree decompositions in practice has been slow.One crucial challenge that is limiting their more widespread use is the computational difficulty of constructing good HDs.In general, finding an HD for a given hypergraph H and width at most k is NP-hard and W [1]-hard when parameterised by k [24] but is tractable when k is fixed, i.e., the problem is in XP in the terminology of parameterised complexity.In fact, a significantly stronger upper bound can be given.Finding an HD of fixed width is in the complexity class LogCFL [24] (contained in NC 2 ), and therefore in theory, highly parallelisable [8].However, the theoretical parallelisability of the problem is demonstrated by construction of an appropriate Alternating Turing Machine [7], and no practical algorithm that allows for effective parallel computation of HDs is known.Here we, to the best of our knowledge, propose the first such algorithm.Related Work.HD computation has received significant attention recently.This is witnessed, for instance, by the development of the large benchmark data set HyperBench [12], novel algorithmic approaches [11,12,26], and being the subject of a recent PACE competition [9].Moreover, a number of new theoretical results [19,20] have been presented, which have deepened our understanding of the problem.Still, the development of a parallel algorithm for hypertree decomposition remains a critical open question.
The two state-of-the-art approaches for computing HDs, det-k-decomp [27] and HtdLEO [31], both rely on techniques that are inherently unsuitable for parallelisation.det-k-decomp is heavily reliant on extensive caching and would therefore require excessive coordination between threads.In HtdLEO, the problem is encoded as an SMT instance and is therefore limited by the lack of parallelisation strategies for SMT solvers.While both algorithms perform well on current benchmarks, their lack of parallelisation ultimately limits them when it comes to solving large instances, i.e., finding HDs of large hypergraphs.This situation is especially disappointing as, on the one hand, single-core performance apparently does not suffice to solve larger instances and, on the other hand, the problem is in fact highly parallelisable in theory.
Interestingly, in Reference [26], a parallel algorithm BalancedGo is proposed for a slightly more general problem of computing generalised hypertree decompositions (GHDs) [25].In HDs, 1:3 the so-called special condition enforces certain constraints on the parent/child nodes in the decomposition tree, and the tree must therefore be treated as rooted.Crucially, this constraint is no longer enforced in GHDs, and it is therefore also no longer necessary to consider the decomposition tree to be rooted.This additional degree of freedom is a key factor in the design of BalancedGo, where it ultimately allows for simple reassembly of individual decompositions of subproblems into a GHD of the full hypergraph.However, this freedom comes at a significant additional computational cost as the corresponding decision problem for computing GHDs is NP-hard even for constant width 2 [20,25] (i.e., it is not even in XP in the parameterised setting).In practice, this leads to an additional exponential factor in the algorithms' complexity in contrast to the complexity of algorithms for computing HDs.
As a final note, we observe that the parallel computation of tree decompositions has been heavily studied (see, e.g., References [5,6,28]).Roughly speaking, these methods intuitively rely on first computing a tree decomposition with width bounded in terms of the width parameter k, but possibly higher than k.In a second step this decomposition is used to find a decomposition of lower width.There is no known analogue to either step for HDs.It is unclear whether techniques for the efficient computation of tree decompositions on graphs can help in the computation of hypertree width in general.Fundamentally, in the hypertree width setting, the cardinality of individual bags in a decomposition is no longer boundable in terms of the width parameter, whereas such a bound is typically key for treewidth techniques.Similarly, using efficient algorithms to compute the treewidth for typical graph encodings of hypergraphs is not helpful in our setting.For example, hyperedges become large cliques in the Gaifman graph, which therefore always has treewidth at least the size of the largest edge (minus 1) and thus is essentially unrelated to the hypertree width.In summary, current approaches either are not amenable to effective parallelisation or compute GHDs and therefore potentially cause exponential additional cost.The goal of this article is to bridge this gap and develop a parallel algorithm for computing hypertree decompositions.
Our Contributions.As argued above, this goal is not achievable by a straightforward extension of current approaches.The two principal algorithms for HDs are inherently unsuited for parallelisation while the parallel algorithm for GHDs fundamentally relies on the fact that GHDs are unrooted.We therefore develop a new theoretical machinery that will allow us to construct HDs in an arbitrary order instead of being limited to a strict top-down or bottom-up construction of the HD.This machinery then allows us to build on some of the ideas of BalancedGo while avoiding the complexity of GHDs.Experimental evaluation demonstrates that the resulting algorithm combines the best of both worlds by scaling effectively with an increase of parallel threads while avoiding the exponential overhead of GHD computation.Our main contributions are as follows: -We develop a new theoretical framework of extended hypergraphs and their balanced separation, and we show that extended hypergraphs always have a balanced separator.To actually find such balanced separators, it is crucial to apply a novel approach that determines pairs of parent and child nodes of an HD (rather than a single node) at a time.-Based on these new results we propose a novel algorithm, log-k-decomp, that searches for balanced separators at arbitrary positions in a potential HD.We argue that our algorithm is well suited for parallelisation; in particular, we prove a logarithmic upper bound on the recursion depth.-We identify a number of further optimisations of our basic algorithm, and we incorporate them into a parallelised reference implementation of log-k-decomp.-We identify the worst-case running time of log-k-decomp as n O (k log(n)) , where k is the width parameter and n the input size.This means that log-k-decomp is, theoretically, not optimal for a problem in the complexity class XP, which the problem of checking whether a hypergraph H has hw ≤ k falls under.However, we show in our experiments that our reference implementation of log-k-decomp is clearly competitive when compared to other systems that implement XP algorithms, such as det-k-decomp.Moreover, in our hybrid approach of combining the splitting into subproblems (whose size is guaranteed to be halved at each step) by log-k-decomp with the sequential det-k-decomp, we are actually back to a worst-case running time of O(n f (k) ).-We compare the performance of log-k-decomp to det-k-decomp and HtdLEO through experiments over the HyperBench benchmark [12].We observe that log-k-decomp outperforms the state of the art significantly.Furthermore, we experimentally verify the parallel scaling behaviour of log-k-decomp.This article is a revised and expanded version of Reference [17].For our revision, we have added Section 6, which provides an illustrative example to better explain how our presented algorithm log-k-decomp works by going through an example run of it.We also added Section 7, which details a number of optimisations and improvements to the base version log-k-decomp, and we prove that these optimisations still allow for a sound and complete algorithm.In addition to this, we have since implemented significant improvements to our proof-of-concept implementation and managed to solve more instances.We detail in the significantly expanded empirical evaluation (Section 8) our improvements to the original implementation and provide updated and expanded results and statistical analyses that show how our implementation fares against the state of the art in computing HDs when running against the standard HyperBench benchmark.
Structure.We formally introduce important concepts and notation in Section 2. The theoretical framework of extended hypergraphs and their balanced separation is established in Section 3. Building on this framework, we introduce the core ideas of the log-k-decomp algorithm and establish a logarithmic bound on its recursion depth in Section 4. The proof details of the correctness of the log-k-decomp algorithm are given in Section 5.The following Section 6 illustrates the algorithm further through a detailed example.We then discuss further ways to optimise the base algorithm in Section 7. The results of our empirical evaluation are presented in Section 8. We conclude with Section 9.

PRELIMINARIES
Conjunctive Queries, Constraint Satisfaction Problems, and hypergraphs.A hypergraph H = (V (H ), E(H )) is a pair consisting of a set of vertices V (H ) and a set of non-empty (hyper)edges E(H ) ⊆ 2 V (H ) .We may assume w.l.o.g. that there are no isolated vertices, i.e., for each v ∈ V (H ), there is at least one edge e ∈ E(H ) with v ∈ e.We can thus identify a hypergraph H with its set of edges E(H ) with the understanding that V (H ) = {v ∈ e | e ∈ E(H )}.A subhypergraph H of H is then simply a subset of (the edges of) H .By slight abuse of notation, we may thus write H ⊆ H with the understanding that E(H ) ⊆ E(H ) and, hence, implicitly also V (H ) ⊆ V (H ).We are frequently dealing with sets of sets of vertices (e.g., sets of edges).For S ⊆ 2 V (H ) , we write S as a short-hand for the union of such a set of sets, i.e., for S = {s 1 , . . ., s }, we have S = i=1 s i .Figure 1 serves as an illustrative example of a hypergraph.
Conjunctive Queries (CQs) are arguably one of the most fundamental types of queries in the database world.Similarly, Constraint Satisfaction Problems (CSPs) are among the most fundamental formalisms in Artificial Intelligence and for modelling combinatorial problems.Formally, both are given by a first-order formula ϕ using only the connectives in {∃, ∧} and disallowing {∀, ∨, ¬}.Given such a formula ϕ, the hypergraph H ϕ corresponding to ϕ is defined as follows: V (H ϕ ) = vars(ϕ), i.e., the variables occurring in ϕ; and E(H ϕ ) = {vars(a) | a is an atom in ϕ}.In the sequel, we will only concentrate on hypergraphs, with the understanding that all results ultimately apply to CQs and CSPs.
Hypertree decompositions and hypertree width.We introduce the used notation first: Given a rooted tree T = N (T ), E(T ) with node set N (T ) and edge set E(T ), we write T u to denote the subtree of T rooted at u, where u is a node in N (T ).Analogously, we write T ↑ u to denote the subtree of T induced by N (T ) \ N (T u ).Intuitively, T u is the subtree of T "below" u and including u, while T ↑ u is the subtree of T "above" u.By slight abuse of notation, we sometimes write u ∈ T instead of u ∈ N (T ) to denote that u is a node in T .Below, we shall introduce node-labelling functions χ and λ, which assign to each node u ∈ T a set of vertices or edges, respectively, from some hypergraph H , i.e., χ (u) ⊆ V (H ) and λ(u) ⊆ E(H ).For a node-labelling function f with f ∈ {χ, λ} and a subtree T of T , we define f (T ) as f (T ) = u ∈T f (u ).
We are now ready to recall the definitions of hypertree decompositions and hypertree width from Reference [24]: An HD D of a hypergraph H = (V (H ), E(H )) is a tuple D = T , χ, λ , such that T = N (T ), E(T ) is a rooted tree, χ and λ are node-labelling functions with χ : N (T ) → 2 V (H ) and λ : N (T ) → 2 E(H ) , and the following conditions hold: (1) for each e ∈ E(H ), there exists a node u ∈ N (T ) with e ⊆ χ (u); The width of an HD D = T , χ, λ is the maximum size of the λ-labels over all nodes u ∈ T , i.e., width(D) = max u ∈T |λ(u)|.Moreover, the hypertree width of a hypergraph H , denoted hw(H ), is the minimum width over all HDs of H . Condition (2) is called the "connectedness condition, " and condition (4) is referred to as the "special condition" in Reference [24].The set χ (u) is often referred to as the "bag" at node u, and we will also call it the "χ -label" of node u.Analogously, the set λ(u) will be referred to as the "λ-label" of u.
If we drop the special condition from the above definition, then we get so-called GHD.The width of a GHD is again defined as the maximum size of the λ-labels over all nodes u ∈ T , and the generalized hypertree width of a hypergraph H , denoted ghw(H ), is the minimum width over all GHDs of H .The problem of checking if an HD of width ≤ k exists, and, if so, computing a concrete HD of width ≤ k is known to be feasible in polynomial time for arbitrarily chosen but fixed k [24].In contrast, for GHDs, this problem has been shown to be NP-complete even if we fix k = 2 [20,25].Indeed, the special condition makes a huge difference when computing a decomposition.Intuitively, its effect is the following: When constructing a GHD or HD topdown, one "guesses" so to speak λ(u) for each node u, starting from the root node.In case of fixed k, there are only polynomially many choices for λ(u).But then, we have to determine also χ (u).By Condition (3), χ (u) must be a subset of λ(u).However, without the special condition, effectively all subsets of λ(u) need to be considered, creating exponentially many choices for χ (u).Ultimately, this is the reason why GHD computation is NP-complete even for k = 2.In contrast, the special condition restricts the possible choices for χ (u) significantly, which makes HD computation tractable.Actually, as we will see in Section 3, it even allows us to define a normal form of HDs in which χ (u) is fully determined when we know λ(u ) of the parent u of u and λ(u).If u is the root, then the special condition simply implies χ (u) = λ(u).
Applications of hypertree decompositions are often formulated in terms of GHDs.As noted above, if we drop the special condition from the definition of HDs, then we obtain precisely the definition of GHDs, and hence every HD is also a GHD.In principle, there are two differences between HDs and GHDs in applications.First, as already discussed, computing HDs is of lower computational complexity than computing GHDs.Second, due to the special condition restricting some possibilities, it is possible that ghw(H ) < hw(H ) [25].However, it is known that hw(H ) will never be much higher than ghw(H ), and indeed it holds that hw(H ) ≤ 3 ghw(H ) + 1 [2], i.e., hypertree width is guaranteed to be at most three times higher than generalized hypertree width.In practice, the situation is even better as there is no solved hypergraph in the standard benchmark Hyperbench [12] where hw(H ) ghw(H ) holds.Hence, for practical purposes, computing HDs can be seen as a more efficient way of computing GHDs.
Throughout this article, we will be dealing with a hypergraph H and a tree T of an HD of H .To avoid confusion, we will consequently refer to the elements in V (H ) as vertices (of the hypergraph) and to the elements in N (T ) as the nodes of T (of the decomposition).

CONNECTION SUBHYPERGRAPHS AND THEIR BALANCED SEPARATION
We introduce here an extension of subhypergraphs and then proceed to define the needed definitions of hypertree decomposition, components, and balanced separation on this new type of extended subhypergraph.
The key idea of our algorithm is to split the task of constructing an HD into subtasks of constructing parts of the HD, which will be referred to as "HD-fragments" in the sequel.These HD-fragments can later be stitched together to form an HD of a given hypergraph.This splitting into HDfragments is realised by choosing a node u of the HD and splitting the HD into one subtree above node u and possibly several subtrees rooted at child nodes of u.The crux of our decomposition algorithm will be that the HD-fragment corresponding to the subtreeT ↑ u above u and the HD-fragments corresponding to the subtrees T u i rooted at the child nodes u i of u can be computed independently, that is, in parallel.However, at the end, these HD-fragments have to be stitched together to form a larger HD-fragment and, ultimately, to form the entire HD of the original hypergraph H .To this end, we have to keep track of, for each HD-fragment, how it connects to other HD-fragments.
The important interface information for the HD-fragment corresponding to the subtreeT ↑ u above node u is the contents of χ (u).We thus introduce the notion of special edges.In case of the HDfragment "above" node u (i.e., the HD-fragment corresponding to the subtree T ↑ u ) this special edge is simply the set χ (u) of vertices.Similarly, for each of the subtrees T u i rooted at the child nodes u i of u, we have to keep track of the interface to χ (u) in the form of a set Conn of vertices, which is the intersection χ (T u i ) ∩ χ (u).

Connection Subhypergraphs and Their HDs
At the heart of our decomposition algorithm in Section 4 will be a recursive function Decomp, which takes as input a subset E of the edges E(H ), a set of special edges Sp, and a set of vertices Conn.

1:7
The goal of Decomp is to construct a fragment of an HD, such that every edge e ∈ E is covered by some node u in the HD-fragment (i.e., e ⊆ χ (u )), all special edges are covered by some leaf node of this HD-fragment (hence, these are the interfaces to the HD-fragments "below"), and Conn must be fully contained in χ (r ) of the root r of this HD-fragment (hence, this is the interface to the HD-fragment "above").Formally, the function Decomp deals with extended subhypergraphs of H in the following sense.

Definition 3.1 (Connection Subhypergraph).
Let H be a hypergraph.An extended subhypergraph with connection interfaces (or connection subhypergraph, for short) of H is a triple E , Sp, Conn with the following properties: -E is a subset of the edge set E(H ) of H ; -Sp is a set of special edges, i.e., Sp ⊆ 2 V (H ) ; -Conn is a set of vertices, i.e., Conn ⊆ V (H ).
Notice that both edges and special edges are sets of vertices.Special edges are, in general, not edges of the hypergraph.They are generated in the course of our decomposition algorithm to keep track of the interface between HD-fragments that may then be constructed independently.To easily refer to the vertices of a connection subhypergraph H = E , Sp, Conn , we shall use the notation V (H ), where we define We now extend several crucial definitions introduced in Reference [24] for hypergraphs to connection subhypergraphs.We shall also introduce a different type of normal form, one that will deviate in one crucial point from the one introduced in Reference [24].

Definition 3.2 (Hypertree Decomposition).
Let H be a hypergraph, and let H = E , Sp, Conn be a connection subhypergraph of H .An HD of H is a tuple T , χ, λ , such that T = N (T ), E(T ) is a rooted tree, and χ and λ are node-labelling functions with λ : N (T ) → 2 E(H )∪Sp and χ : N (T ) → 2 V (H ) , such that the following conditions hold: (1) for each u ∈ N (T ): Clearly, H can also be considered as a connection subhypergraph of itself by taking the triple E(H ), ∅, ∅ .Then the HDs of the connection subhypergraph E(H ), ∅, ∅ and the HDs of hypergraph H coincide.
The ultimate goal of the above definition of an HD of a connection subhypergraph of some hypergraph H is to capture fragments of an HD of a hypergraph H , which can be stitched together to actually yield an HD of the entire hypergraph H .This is also the reason why, in Condition (1) above, edges in λ(u) may be chosen from the entire set E(H ) of edges in H .But ultimately, the "task" of an HD-fragment is to cover all edges and special edges of H .That is why in Conditions (2) and (3) only E (and not E(H )) is mentioned.
We note that we are slightly sloppy in using the terms "fragment of an HD" or, synonymously, "HD-fragment": In the first place, we thus mean a part of an HD of the original hypergraph H .The goal of our decomposition algorithm is to split the task of constructing an HD of H into pieces and to construct such HD-fragments.However, as mentioned above, we need to keep track of the interfaces between various HD-fragments.In particular, special edges are introduced to keep track of the interface between an HD-fragment above some node u in the final HD and the HDfragments below this node u.Therefore, we also refer to an HD that consists of a part of the final HD plus possibly some leaf nodes that cover special edges (i.e., that contain the aforementioned interface information) as an "HD-fragment." Actually, HDs of the connection subhypergraph that we consider in our decomposition algorithm are such HD-fragments, i.e., they contain a part of an HD of the original hypergraph H plus possibly leaf nodes that cover the special edges of H . Example 3.3.An example of a connection subhypergraph is shown in Figure 2, as well as an HD of this connection subhypergraph.We can see that no λ-label uses more than two hyperedges, and thus this HD has width 2, and the hw of the hypergraph is ≤ 2. In fact, the hypergraph contains alpha cycles [10], e.g., {e 2 , e 3 , e 4 , e 5 }.Hence, we also know that its hw must be > 1.Taken together, its hw is therefore exactly 2.
In Reference [24], Definition 5.1, a normal form of HDs was introduced.In Definition 3.8, we will carry the notion of normal form over to HDs of connection subhypergraphs.To this end, it is convenient to first define the set of (possibly special) edges covered for the first time (in top-down direction) by some node or by some subtree of an HD.Definition 3.4.Let H = E , Sp, Conn be a connection subhypergraph of some hypergraph H , and let D = T , χ, λ be an HD of H .For a node u ∈ T , we write cov(u) to denote the set of edges and special edges covered for the first time at u, i.e., cov(u and for all ancestor nodes u of u, f χ (u ) holds}.For a subtree T of T , we define cov(T ) = u ∈T cov(u).
Example 3.5.To illustrate the function cov, we shall give some example evaluations of it when applied to the HD given in Figure 2. Clearly, any (special) edge covered at the root node u 1 is covered for the first time in an HD, so we have cov(u 1 ) = {e 1 , e 6 }.For the internal node u 4 , we have cov(u 4 ) = {e 4 , e 3 }.So we see that the special edge e 3 is actually covered for the first time in u 4 .The node u 5 is only needed to satisfy Condition (2) of Definition 3.2, where we require that each special edge must appear in a node with a single edge in the edge cover, namely the special edge itself.
Components of a hypergraph or, more generally, of a connection subhypergraph, play a crucial role in the construction of a decomposition and also for the definition of our normal form for HDs of a connection subhypergraph.-We define [U ]-adjacency as a binary relation on E ∪Sp such that two (possibly special) edges Example 3.7.An example for a separator that generates multiple connected components can be seen in Figure 3.The separator S is the union of e 2 ∪e 6 , marked via thicker edges.The corresponding [S]-components C 1 = {e 3 , e 4 , e 5 } and C 2 = {e 1 , e 7 , e 8 } are highlighted visually.Note that edges fully covered by S (in our example, e 2 and e 6 ) are not part of any [S]-component.
Let S be a set of edges and special edges with U = S. Then we will also use the terms [S]connectedness and [S]-components as a short-hand for [U ]-connectedness and [U ]-components, respectively.

Normal Form of HDs of Connection Subhypergraphs
Analogously to the normal form of HDs of hypergraphs in Reference [24], we now introduce a normal form of HDs of connection subhypergraphs.

Definition 3.8 (HD-normal Form of connection subhypergraphs)
. Let H = E , Sp, Conn be a connection subhypergraph of some hypergraph H , and let D = T , χ, λ be an HD of H .We say that D is in normal form if for every node p in T and every child node c of p the following properties hold: ( We will show in Claim A of the proof of Theorem 3.10 that, in any HD of H , cov(T c ) is the union of one or several [χ (p)]-components.Condition 1 of the normal form requires something stronger, namely that cov(T c ) is exactly one Condition 2 serves to eliminate redundant nodes from an HD.As will be seen in the "Condition 2"-part of the proof of Theorem 3.10, if there is no such f ∈ cov(T c ) with f ⊆ χ (c), then χ (c) ⊆ χ (p), and we can actually delete node c from the HD.
Condition 3 is the only place where we deviate from the normal form in Reference [24].The purpose of Condition 3 in Reference [24] is to make sure that χ (c) is uniquely determined whenever λ(c), χ (p), and cov(T c ) are known.However, there also would have been other choices to achieve Fig. 4.An example for a non-normal form HD. This HD decomposes the connection subhypergraph given earlier in Figure 2.
this goal.Our Condition 3 chooses χ (c) minimally.That is, to ensure the special condition, χ (c) must contain all vertices from λ(c) that occur in χ (T c ). Since, by definition, all edges in cov(T c ) are covered at some node in T c , all vertices from λ(c) ∩ cov(T c ) must occur in χ (c).However, as will become clear in the proof of Theorem 3.10, there is no need to add further vertices to χ (c), since vertices not occurring in cov(T c ) can never violate the connectedness condition at node c as long as we stick to our strategy of choosing χ (u) minimally also for all nodes u ∈ T c .In contrast, Condition 3 in Reference [24] chooses χ (c) maximally.That is, also all vertices in λ(c) that occur in χ (p) are added to χ (c).This deviation from the normal form in Reference [24] is crucial for the design of our algorithm, since, in our construction of an HD, we will be able to derive the possible sets cov(T c ) as soon as we have λ(p) and λ(c), but we will "know" χ (p) only much later in the algorithm.
In the sequel, to improve readability, we will often use the name C p when referring to the unique [χ (p)]-component of H from Condition 1 of Definition 3.8, which coincides with cov(T c ).
Example 3.9.We already saw an example of a normal form HD in Figure 2 with the connection subhypergraph right next to it.We shall call this connection subhypergraph H = E , Sp, Conn , where E = {e 1 , e 2 , e 4 , e 5 , e 6 , e 7 , e 8 }, Sp = {e 3 }, and Conn = { f , д, h}.This connection subhypergraph in turn is based on the hypergraph we saw earlier in Figure 1.To illustrate the differences between normal form and non-normal form HDs, we give an example of an HD that violates the normal form stated in Definition 3.8 in Figure 4. To understand why it does not satisfy Definition 3.8, we note that there are exactly two [χ u 1 ]-components of H , and yet u 1 has only one child node.Condition (1) requires for each [χ u 1 ]-component of H to be covered in a separate subtree, whereas the non-normal form HD covers both components in the same subtree.
We now carry over two key results from Reference [24], whose proofs can be easily adapted to our setting of connection subhypergraphs.Theorem 3.10 (cf.Reference [24], Theorem 5.4).Let H be a connection subhypergraph of some hypergraph H , and let D be an HD of H of width k.Then there exists an HD D of H in normal form, such that D also has width at most k.
Proof.The proof proceeds in several steps: Claim A (cf. Reference [24].Lemma 5.2).Let p, c be nodes in the HD D such that p is the parent of c.Moreover, let C be a Proof of Claim A. We proceed by contradiction.Suppose that C ∩ cov(T c ) ∅, but C cov(T c ) holds.That is, there exist (possibly special) edges e, e ∈ C such that e ∈ cov(T c ) and e cov(T c ), i.e., e is covered by some nodeu ∈ N (T ) outsideT c .However, e, e are [χ (p)]-connected, i.e., there exists a sequence of (possibly special) edges e = e 1 , e 2 , . . ., e = e such that any two neighbouring edges e i , e i+1 are [χ (p)]-adjacent.By assumption, e 1 is covered by some node outside T c , and e is covered by some node in T c .Hence, there must exist edges e i , e i+1 such that e i is covered by some node outside T c and e i+1 is covered by some node in T c .But e i and e i+1 are [χ (p)]adjacent, i.e., they share a vertex x that is not in χ (p).So e i is actually covered by a node outside T c that is different from p. Hence, vertex x violates the connectedness condition of HDs (Condition 3 of Definition 3.2)-a contradiction.
Claim B (cf. Reference [24].Lemma 5.3).Let p be a node in the HD D.Moreover, let C be a [χ (p)]-connected set of edges and special edges of H , and let V = C \ χ (p).Moreover, for any set of vertices X ⊆ V (H ), let nodes(X ) denote the set of nodes u in the HD D with χ (u) ∩ X ∅.Then nodes(V ) forms a connected subtree of D.
Proof of Claim B. We proceed by induction of the number of edges in C. First, consider a single edge e.By the connectedness condition of HDs (Condition 3 of Definition 3.2), we must have that, for every x ∈ e \ χ (p), nodes({x }) induces a connected subtree of D.Moreover, by Condition 2 of Definition 3.2, there exists a node u in D with e ⊆ χ (u).Hence, u ∈ nodes({x }) for every x ∈ e \ χ (p), and, therefore, also nodes(e \ χ (p)) forms a connected subtree of D.
For the induction step, it suffices to show that, for any two (possibly special) edges e, e , if e, e are [χ (p)]-adjacent, then nodes((e ∪e )\ χ (p)) forms a connected subtree of D. We have just shown that nodes(e \ χ (p)) and nodes(e \ χ (p)) form connected subtrees of D.Moreover, since e, e are [χ (p)]-adjacent, they share a variable x χ (p).By Condition 3 of Definition 3.2, nodes({x }) is a connected subtree of D. Hence, this latter subtree connects the two subtrees nodes(e \ χ (p)) and nodes(e \ χ (p)) and, therefore, also nodes((e ∪ e ) \ χ (p)) forms a connected subtree of D.
We are now ready to show that any HD can be transformed into an HD in normal form without increasing the width.This transformation proceeds top-down.Consider an arbitrary HD D = (T , χ, λ) of a connection subhypergraph H . Let (p, c) be a pair of nodes in D that violates one of the Conditions 1-3 of the normal form, and suppose that p is highest up in the tree T with this property.Then we can transform D as follows to ensure that (p, c) satisfies the Conditions 1-3.Condition 1.By Claim A, we have that cov(T c ) is the union of one or several [χ (p)]-components.By Condition 1 of the normal form, there is exactly one [χ (p)]-component C p of H satisfying C p ⊆ cov(T c ) and, therefore, C p = cov(T c ). Suppose that this is not the case, i.e., there exist . By Claim B, nodes(V i ) forms a connected subtree of T c .Let T i denote this subtree.We take an isomorphic copy T i of T i .For every node u ∈ N (T i ), we write u to denote the corresponding node in N (T i ) under the isomorphism between T i and T i .Then we define labelling functions χ i and λ i on the nodes in T i as follows: For every u ∈ N (T i ), we set χ i (u ) = χ (u) ∩ C i and λ i (u ) = λ(u).Now we transform D as follows: We delete the entire subtree T c from T .Instead, we append the trees T 1 , . . .,T as new subtrees immediately below p, i.e., the roots r 1 , . . ., r of the trees T 1 , . . .,T become new child nodes of p. Clearly, every pair (p, r i ) satisfies Condition 1 of the normal form, i.e., there exists a [χ (p)]-component C p of H such that C p = cov(T c ), namely C p = C i .Moreover, it is easy to verify that none of the Conditions 1-5 of Definition 3.2 is violated by this transformation.Condition 2. Suppose that (p, c) is a pair of nodes in D that satisfies Condition 1 but violates Condition 2 of the normal form.Let C p denote the [χ (p)]-component with cov(T c ) = C p .By the violation of Condition 2, there is no f ∈ C p with f ⊆ χ (c).Note that then there is also no f ∈ C p with f ∈ λ(c).Indeed, suppose to the contrary that there were some f ∈ C p with f ∈ λ(c).Then, by the special condition of HDs (Condition 4 of Definition 3.2), all vertices in f would have to be in χ (c), and Condition 2 of the normal form would not be violated by (p, c).
It follows that λ(c) contains only edges and special edges that are covered outside T c .Hence, by the connectedness condition of HDs (Condition 3 of Definition 3.2), we must have χ (c) ⊆ χ (p).But then we may simply delete the node c from T and turn all child nodes of c into child nodes of p. Clearly, this does not lead to a violation of any of the conditions of the definition of HDs.C p ⊆ χ (c).We can thus perform the following operation on the entire subtree T c : All vertices from χ (T c ) that are outside C p shall be removed from the χ -labels of the nodes in T c .That is, for any n ∈ T c , we may set From the reasoning above, it follows that this may not lead to any violations of the conditions of Definition 3.2.

Properties of Connection Subhypergraphs and Their Normal Form HDs
We move on to showing two key properties of connection subhypergraphs.First, we show that property (1) of Definition 3.2 can be strengthened.Not only is there always an HD such that cov(T c ) is a [χ (p)]-component, but in fact there is also an HD such that cov(T c ) is always a [λ(p)]component.Second, we show that a connection subhypergraph can always be separated in a balanced fashion by a bag of one of its HDs.
Toward the first goal we first show the following intermediate lemma.
Lemma 3.11 (cf.Reference [24], Lemma 5.8).Let H be a connection subhypergraph of some hypergraph H and let D = T , χ , λ be an HD in normal form of H .Moreover, let p, c be nodes in T such that p is the parent of c and let -connected, and it is maximal with this property.Clearly, by χ (c) ⊆ λ(c).C c is also [χ (c)]-connected.It remains to show that it is maximal with this property.We proceed by contradiction.Suppose to the contrary that C c is not maximally [χ (c)]-connected.Then there exists a (possibly special) edge e ∈ C p \C c such that e is [χ (c)]-adjacent to some e ∈ C c .By Claim A, then e and e are also [λ(c)]-adjacent.However, this contradicts the assumption that The principal proof ideas of Theorem 3.10 and Lemma 3.11 are the same as for the corresponding results (Theorem 5.4 and Lemma 5.8) in Reference [24].The main technical difference between the proofs here and there comes from the fact that we have defined [U ]-components in Definition 3.6 as sets of edges and special edges while they are defined as sets of vertices in Reference [24].We have made this decision in order to simplify the presentation of our decomposition algorithm in the next section.In addition, some of the proof arguments for Theorem 3.10 and Lemma 3.11 could be slightly simplified compared with the corresponding results in Reference [24].
As has been noted above, our deviation from Reference [24] in the definition of the χ -label of nodes in a normal form HD is essential for the design of our decomposition algorithm to be presented in the next section.However, as far as the proof arguments of Theorem 3.10 and Lemma 3.11 are concerned, this deviation is inessential, since the "downward" components in an HD are not affected by adding or removing vertices from the χ -label of the parent node to the χ -label of the child node.However, for our purposes, we need a slightly stronger version of the above lemma: Recall that the HD construction in Reference [24] proceeds in a strict top-down fashion.Hence, when dealing with λ(c), the bag χ (p) is already known.This is due to the fact that, initially at the root r , we have χ (r ) = λ(r ) by the special condition.And then, whenever λ(c) is determined and χ (p) plus a [χ (p)]-component are already known, χ (c) can also be computed.However, in our HD algorithm, which "jumps into the middle" of the HD to be constructed, we do not automatically have χ (p) available when determining λ(c).Hence, we need to slightly extend the above lemma to the following corollary, which allows us to identify [χ (p)]-components with [λ(p)]-components also for the parent node p. Corollary 3.12.Let H be a connection subhypergraph of some hypergraph H and let D = T , χ, λ be an HD in normal form of H .Moreover, let p, c be nodes in T such that p is the parent of c.Then, the following property (1') holds: There exists a Proof.We distinguish the two cases as to whether p is the root node of T or not.Root node.Let p = r .Moreover, let c be a child node of r .We know, by the definition of the normal form, that cov( To this end, we first show (analogously to the proof of Claim A in the proof of Lemma 3.11) that two edges e, e ∈ cov(T c ) are [χ (r )]-adjacent if and only if they are [λ(r )]-adjacent.
Node different from the root.Consider an arbitrary node p in T and let c be a child of p.Moreover, let p be the parent node of p.By the definition of the normal form, we know that cov( In Reference [12], balanced separators were used to design an algorithm for GHD computation.Below, we formally define balanced separators for our notion of connection subhypergraphs and we show that such a balanced separator always exists.

Definition 3.13 (Balanced Separators).
Let H be a connection subhypergraph of some hypergraph H , and let D = T , χ, λ be an HD of H .A node u of T is a balanced separator if the following holds: -for every subtree T u i rooted at a child node u i of u, we have |cov( Intuitively, this means that none of the subtrees "below" u covers more than half of the edges of E ∪ Sp and the subtree "above" u even covers less than half of the edges of E ∪ Sp.Lemma 3.14.Let H be a connection subhypergraph of some hypergraph H , and let D = T , χ, λ be an HD of H . Then there exists a balanced separator in D.
Proof.We show that, given an arbitrary HD, we can always find a balanced separator as follows: Initially, we set u = r for the root node r of T and distinguish two cases: holds for every subtree T u i rooted at a child node u i of u, then u is a balanced separator and we are done.Otherwise, there exists a child node holds for the subtree T u i rooted at u i .Of course, there can exist only one such child node u i .Moreover, by holds for every subtree T u i rooted at a child node u i of u, then u is a balanced separator and we are done.Otherwise, there exists a child node holds for the subtree T u i rooted at u i .Again, there can only be one such u i .So we set u = u i and iterate the same considerations.This process is guaranteed to terminate, since, eventually, we will reach a leaf node of T .

THE LOG-K-DECOMP ALGORITHM
Algorithm log-k-decomp, whose pseudo-code description is shown in Algorithm 1, aims at constructing an HD in normal form according to Definition 3.8 of width ≤ k for a given hypergraph H and integer k ≥ 1. W.l.o.g., we assume that H is connected, since, otherwise, we can always compute an HD for each connected component of H separately and then construct an HD of H by combining the HDs of the connected components in an arbitrary way, e.g., choose the HD of one connected component and append the root node of any other HD as additional child of arbitrary nodes of the chosen HD.It is easy to verify that no condition of the definition of HDs can thus be violated.
The task of constructing an HD is split into subtasks that can then be processed in parallel.The key idea of function Decomp is to construct an HD of H by finding a balanced separator (referred to as node c in Algorithm 1) of the HD to be constructed and to partition the set of edges and special edges of H via this balanced separator.The HD construction then proceeds by recursively calling function Decomp for such subsets.More formally, let C 1 , . . .,C m denote all [χ (c)]-components of H and let C 0 = {e ∈ H .E ∪ H .Sp | e ⊆ χ (c)}.Then H .E ∪ H .Sp can be partitioned into the sets C 0 , . . .,C m .For complexity reasons, our algorithm aims at finding an appropriate value of λ(c) rather than χ (c).Clearly, for bounded width k of the desired HD, there are only polynomially many candidates for λ(c) while there are, in general, exponentially many candidates for χ (c).By Corollary 3.12, for an HD in normal form, all [χ (c)]-components of H covered by a subtree rooted at a child of c are also [λ(c)]-components.But now we face two problems: (1) Which components of H are covered by a subtree rooted at a child of c (as opposed to components that are covered by some node above the balanced separator c) and ( 2) how can we determine C 0 , i.e., the edges and special edges covered by χ (c)?
We solve these problems by also searching for the λ-label of the parent node p of c.Indeed, in an HD in normal form, cov(T c ) corresponds to a [χ (p)]-component C p (referred to as comp down in Algorithm 1).By Corollary 3.12, C p is also a [λ(p)]-component.This allows us to solve the first problem, since the [λ(c)]-components that are covered by a subtree rooted at a child of c are precisely those [λ(c)]-components that are a subset of C p .Knowing C p = cov(T c ) also allows us to solve the second problem in a very simple way: We know that cov(T c ) is partitioned into the (possibly special) edges covered by χ (c) and the (possibly special) edges contained in one of the [χ (c)]-components inside C p .Hence, C 0 consists of all (possibly special) edges in cov(T c ) not contained in one of the [χ (c)]-components inside C p .In function Decomp, the set C 0 is only dealt with implicitly in that we call Decomp recursively for all [χ (c)]-components inside C p (on line 28) and for the connection subhypergraph containing all edges and special edges of H outside C p (on line 33).In other words, the edges and special edges of H that are not contained in one of the [χ (c)]-components inside C p and that are outside C p (that is, the edges and special edges covered by χ (c)) are considered as done and are not contained in any recursive call to Decomp.
We now explain informally the various steps of function Decomp.A formal proof of the correctness of Algorithm 1 (in particular, of function Decomp) will be provided in Section 5, and an example will be worked out in detail in Section 6.The base case of function Decomp is reached (lines 5-8) when the existence of such an HD-fragment is trivial, i.e., either there are at most k edges and no special edges left or there is no edge and only one special edge left.In these cases, the desired HD-fragment simply consists of a single node whose λ-label either consists of the ≤k edges or of the single special edge, respectively.
Function Decomp is controlled by two nested loops (lines 9-35 for the outer loop and lines 17-35 for the inner loop), which search for the λ-labels of two adjacent nodes p and c of the desired HDfragment, such that p is the parent and c is the child.The idea of determining two nodes p and c has been explained above.In particular, we want node c to be a balanced separator of the connection subhypergraph H .By Lemma 3.14, a balanced separator is guaranteed to exist.To find a balanced separator c, we have to make sure that node c satisfies the two conditions of Definition 3.13, i.e., (1) all of the subtrees rooted at a child of c cover at most half of the edges and special edges in H , and (2) the subtree T ↑ c "above" c covers strictly fewer than half of the edges and special edges in H .For the second condition, observe that comp down (chosen at line 12) is meant to be covered precisely by T c .Due to the fact that we are searching for an HD in normal form, we may assume that T c covers exactly one [χ (p)]-component comp down , which, by Corollary 3.12, is also a [λ(p)]component.Further observe that the edges and special edges covered by T ↑ c and the set comp down partition the edges and special edges in H . Hence, checking if comp down contains more than half of H (on line 11) is equivalent to checking condition (2), i.e., T ↑ c covers strictly fewer than half of the edges and special edges in H .To check that c also satisfies the first condition of Definition 3.13, we have to compute all [λ(c)]-components inside comp down (line 22) and check that the size of each of them is at most half of the size of H (line 23).Again, since we are only interested in HDs in normal form, we may assume here that each subtree rooted at a child of c covers exactly one of these [λ(c)]-components.
If such a balanced separator λ(c) together with the λ-label λ(p) at the parent node p of c has been found, then several checks have to be performed to make sure that the HD-fragment under construction satisfies the connectedness condition.For instance, all vertices in the intersection of Conn (i.e., the interface of the HD-fragment currently being constructed with the remaining HD "above" this HD-fragment) with component C p (i.e., a component "below" node p) also have to occur in λ(p) (line 15).
Suppose that all these checks succeed.From λ(p) and λ(c), we can compute χ (c) according to Condition 3 of the normal form introduced in Definition 3.8 (line 18).In the HD D to be constructed for the connection subhypergraph H , the edges and special edges of H can be split into three disjoint categories as follows: (1) the edges and special edges covered by χ (c), (2) the edges and special edges covered by a subtree rooted at some child node of c, and (3) the edges and special edges covered in the HD "above" c.In the call for component C i , the interface Conn i is obtained simply as the intersection of the vertices in C i and in χ (c) (line 26, where C i is referred to as x).All of the remaining [χ (c)]-components are taken care of by the HD-fragment "above" c, which we try to construct in another recursive call of function Decomp (line 33).In this recursive call, χ (c) is added as yet another special edge-in addition to the edges and special edges in the [χ (c)]-components outside C p .The additional special edge in the recursive call for the HD-part "above" node c and the interfaces Conn defined for each of the components as the intersection against χ (c), in the recursive calls for the HD-parts "below" node c ensure that we can (provided that all recursive calls of function Decomp are successful) stitch together the HD-fragments of these recursive calls to an HD-fragment of the connection subhypergraph H of H .
To summarize, if all recursive calls return "true, " then the overall result of this call to function Decomp is successful and returns "true" (line 35).If at least one of the recursive calls returns "false, " then we have to search for a different label λ(c) (in the next iteration of the "ChildLoop").If eventually all candidates for λ(c) have been tried out and none of them was successful, then we have to search for a different label λ(p) of the parent node p (in the next iteration of the "ParentLoop") and restart the search for λ(c) from scratch.Only when also all candidates for λ(p) have been tried out and none of them was successful does function Decomp return the overall result "false" (line 36).
We conclude our informal discussion of Algorithm 1 by mentioning a slightly subtle point concerning the candidates for λ(p) in the "ParentLoop." By the condition 0 ≤ |λ p | ≤ k on line 9, λ p = ∅ is also considered as a possible candidate.Of course, in the HD D to be constructed for the connection subhypergraph H , we do not want to have nodes with empty λ-label and, hence, also empty χ -label.Nevertheless, this case is included for technical reasons.Actually, the check on line 15 only succeeds if H .Conn is also empty.We are assuming that the input hypergraph H is connected.Hence, H .Conn = ∅ can only occur when the HD D to be constructed by the current call of Decomp contains the root of the HD D of H .In this case, p is a "dummy" parent of node c, and c will ultimately be the root of D and, therefore, also of D. Indeed, the effect of λ p = ∅ is that comps p on line 10 contains a single [λ p ]-component, namely H .E ∪ H .Sp.Consequently, the recursive call on line 33 for the component "above" node c is with a connection subhypergraph that contains no edge and only one special edge, namely χ (c).This means that the corresponding HD-fragment implicitly constructed for this call of Decomp consists of a single node u with λ(u) = {χ (c)} and χ (u) = χ (c).When assembling the HD D of H from the HDs implicitly constructed by the various recursive calls of function Decomp, this node u gets merged with the root c of the rest of the HD D of H and, thus, disappears.The details of the construction of an HD from a successful call of function Decomp will be worked out in detail in the soundness proof in Section 5.
Below, we state the crucial property of log-k-decomp, which makes this approach particularly well suited for a parallel implementation.Proof.Theorem 4.1 has several parts.Below, we prove that the recursion depth is bounded logarithmically in the number of edges of H .The correctness of the algorithm log-k-decomp as well as the polynomial-time upper bound on the construction of an HD in case of a successful run of the algorithm (i.e., if it returns "true") will be proved in a separate section, namely Section 5.
The size of the connection subhypergraph in the call of function Decomp in the main program can only be bounded by the size of H itself.However, in every subsequent execution of Decomp for some connection subhypergraph (H .E, H .Sp, H .Conn), we always choose node c as a balanced separator.By Lemma 3.14, such a balanced separator always exists.The connection subhypergraphs in the recursive calls are therefore guaranteed to have size at most . Note that the rounding up is necessary, because the new special edge χ (c) is added in the recursive call for the HD-fragment above c.Without this special edge, this component is guaranteed to be strictly smaller than . At any rate, also with the upper bound on the size of the connection subhypergraphs of H in the recursive calls, we thus get an upper bound O(log(|H |)) on the recursion depth.
Note that we have formulated algorithm log-k-decomp as a decision procedure that decides if hw(H ) ≤ k holds for given H and k.In case of a successful computation (i.e., return-value "true") it is easy to assemble a concrete HD of width ≤ k of H from the HD-fragments corresponding to the various calls of procedure Decomp.In Section 5, we shall discuss the construction of such a concrete HD as part of the correctness proof of Algorithm 1.
We emphasize two further important properties of algorithm log-k-decomp: First, it should be noted that the logarithmic bound on the recursion depth does not restrict the form of the HD in any way.In particular, it does not imply a logarithmic bound on the depth of the HD.The bound on the recursion depth is achieved by our novel approach of constructing the HD by recursively "jumping" to a balanced separator of the HD-fragment to be constructed rather than constructing the HD in a strict top-down manner as proposed in previous approaches [24,27].
Fast Parallel Hypertree Decompositions 1:19 Second, as was explained at the beginning of this section, it is crucial in our approach that we search for appropriate λ-labels for a pair (p, c) of nodes, where p is the parent of c.One of the reasons why we need the λ-label of the parent is that this allows us to determine χ (c) from λ(c).And only when we know χ (c) we can be sure which edges are indeed covered by χ (c).This knowledge is crucial to guarantee that all of the recursive calls of function Decomp have to deal with a connection subhypergraph whose size is halved, which in turn guarantees the logarithmic upper bound on the recursion depth.This strategy is significantly different from all previous approaches of decomposition algorithms.In References [12,26], a parallel algorithm for generalised hypertree decompositions is presented.There, the problem of determining the χ -label of the balanced separator is solved by adding a big number of subedges to the hypergraph so that one may assume that χ (u) = λ(u) holds for every node u.Clearly, this addition of subedges, in general, leads to a substantial increase of the hypergraph.In Reference [4], a preliminary attempt to parallelise the computation of HDs was made without handling pairs of nodes.However, in the absence of λ(p), we cannot determine χ (c) from λ(c).Consequently, we do not know which edges covered by λ(c) are ultimately covered by χ (c).Hence, all the edges covered by λ(c) would have to be added to the recursive call of Decomp for the HD-part "above" c, thus destroying the balancedness and the logarithmic upper bound on the recursion depth.
By Theorem 4.1, Algorithm log-k-decomp guarantees a logarithmic bound on the recursion depth and thus provides a good basis for a parallel implementation.Nevertheless, it still leaves room for several improvements.For instance, we can define also negative base cases to detect the overall answer "false" faster, we can restrict the edges that may possibly be used in the λ-labels of a connection subhypergraph (and provide them as an additional parameter of the function Decomp), and so on.These ideas and several further improvements-together with the pseudo-code of the resulting improved algorithm-are presented in Section 7.
The complexity of log-k-decomp.It is known that deciding whether hw(H ) ≤ k holds for a given hypergraph H is in XP in terms of parameterised complexity, i.e., there is an O(n f (k) ) algorithm that checks whether a hypergraph with size n has width ≤ k.However, our algorithm log-k-decomp does not achieve this bound and has a worst-case running time of only n O (k log n) : Each call of Decomp iterates through roughly n 2k separator pairs.We cannot significantly bound the number of balanced separators and thus each recursion incurs another multiplicative factor in the order of n 2k in the worst case.Since recursion can nest up to log n times, we observe the stated worst-case complexity.
The additional log factor in the exponent is an indirect consequence of our use of special edges.The canonical XP algorithm for deciding hw ≤ k [24] is stated in terms of a log-space Alternating Turing Machine (ATM) [7] (wrt.fixed k).It is known that log-space ATMs can be simulated by polynomial time deterministic Turing Machines [30].However, this simulation relies on caching to avoid repetition of computation trees (cf. the NewDetKDecomp algorithm in Reference [12]).Notably, this cache is always of polynomial size due to the log-space bound on the representation of each ATM state.In our algorithm, keeping track of special edges is required to guarantee that decompositions for individual components fit together.Intuitively, this book-keeping of special edges makes the potential number of different computation trees exponential (in order log n) and thus on a theoretical level, caching is no longer sufficient to achieve XP running time.
However, our empirical evaluation in Section 8 confirms that these concerns are mainly of theoretical nature as our algorithm clearly outperforms NewDetKDecomp from Reference [12], which is strictly in XP.In practice, we also employ a hybrid strategy, where we switch to the simpler NewDetKDecomp algorithm after a sufficient number of balanced splitting steps by log-k-decomp.
In Section 8, we will see that this performs best in practice as the benefit of balanced splitting diminishes once the connection subhypergraphs become small.Such a hybrid method in fact exhibits XP worst-case complexity if we switch to NewDetKDecomp at a constant depth d (leading to no more than n O (kd) calls of the n O (k) algorithm NewDetKDecomp).At any rate, in our implementation, we choose a more sophisticated hybridisation strategy (see Section 8.3) for better practical performance than the constant depth criterion.log-k-decomp vs. BalancedGo.We conclude this section by pointing out the main difference between our new log-k-decomp algorithm for HD computation and the BalancedGo algorithm from Reference [26] for GHD computation.Both algorithms are based on "guessing" a balanced separator, computing components relative to this separator, and then recursively decomposing each component.However, as mentioned in Section 2, in case of GHDs, we do not have the special condition (Condition 4 in the definition of HDs recalled in Section 2) at our disposal to restrict the possible values of χ (u) when we know λ(u).Hence, a different method for getting access to χ (u) and, in the next step, to the [χ (u)]-components is required.Indeed, such a method was proposed in Reference [20] and then refined in Reference [26]: By adding certain subedges of the edges in E(H ), we may assume w.l.o.g. that χ (u) = λ(u) holds.That is, the hypergraph H = (V (H ), E(H )) is transformed into a new hypergraph H = (V (H ), E (H )) such that, for every e ∈ E (H ), there exists e ∈ E(H ) with e ⊆ e.In References [20,26] a way to restrict the set E (H ) is proposed so that we do not have to add all of 2 E(H ) .Nevertheless, in general, E (H )\E(H ) is still exponentially big.Of course, by the NP-hardness of GHD-computation even for fixed k = 2, this exponential blow-up is to be expected.This is in sharp contrast to our new log-k-decomp algorithm for HD-computation, where we never need to add any edges to the hypergraph.However, in log-k-decomp, we always have to "guess" λ(p) of the parent p in addition to the balanced separator λ(c), as was explained at the beginning of this section.
A minor technical difference comes from the fact that, for HDs, the definition of the root node is essential (due to the "special condition, " i.e., Condition 4 of Definition 3.2), whereas a GHD can be rooted at any node.Therefore, in log-k-decomp, the interface of an HD-fragment to the neighbouring fragment above (i.e., the set Conn of vertices in our algorithm) and the interface to the neighbouring fragments below (i.e., the special edges) are treated differently.Consequently, in the splitting step of log-k-decomp, we add a special edge only to the recursive call for the component "above" node c.In contrast, BalancedGo treats the interfaces to all neighbouring GHDfragments equally and, therefore, adds a special edge to each component resulting from a splitting step.Adding a special edge only to 1 recursive call in case of log-k-decomp (as opposed to adding a special edge to every recursive call in case of BalancedGo) further increases the complexity advantage of log-k-decomp over BalancedGo.
To conclude, the big advantage of log-k-decomp compared with BalancedGo is that log-k-decomp does not add any edges to the hypergraph while BalancedGo, in the worst case, has to add exponentially many new edges.The price to pay for this by log-k-decomp is that we always need to "guess" the λ-labels of pairs (p, c), where c is the balanced separator and p its parent.In BalancedGo, due to the added subedges, only λ(c) needs to be "guessed." As will be seen in the experimental results in Section 8, the advantage of not having to add subedges clearly outweighs the disadvantage of having to deal with pairs (p, c) of nodes.

CORRECTNESS PROOF OF LOG-K-DECOMP
We prove the soundness and completeness of the algorithm log-k-decomp given in Algorithm 1 separately.The polynomial-time upper bound on the construction of an HD in case of a successful run of the algorithm (i.e., if it returns "true") will be part of the soundness proof.
It remains to consider the call of function Decomp on line 33.Suppose that comps c on line 22 is of the form comps c = {x 1 , . . ., x }.By the definition of components in Definition 3.6, H (that is, H .E ∪ H .Sp) can be partitioned into the following disjoint subsets: - . By construction (line 22), all components x i are contained in comp down .Hence, we actually have V (H ) = V (comp down ) ∪ χ (c) ∪ V (z).The recursive call of function Decomp on line 33 is with the edges and special edges in z plus χ (c) as an additional special edge.Hence, we have V (comp up ) = V (z) ∪ χ (c) when Decomp is called on line 33 with parameter H comp up .It is therefore sufficient to show that H .Conn ⊆ V (z) ∪ χ (c) holds.
By the induction hypothesis, we may assume that H .Conn ⊆ V (H ) holds.The check on line 15 ensures that V (comp down )∩H .Conn ∩ ⊆ λ(p) holds for the edge set λ(p).Moreover, the check on line 19 ensures that ( λ(p))∩V (comp down ) ⊆ χ (c).In total, we thus have H .Conn ∩V (comp down ) ⊆ χ (c).Together with V (H ) = V (comp down ) ∪ χ (c) ∪ V (z) and H .Conn ⊆ V (H ), we may thus conclude H .Conn ⊆ V (z) ∪ χ (c) and, therefore, H .Conn ⊆ V (comp up ) (on line 33).Hence, also the call of function Decomp on line 33 satisfies Claim A. Soundness Proof.Suppose that algorithm log-k-decomp returns "true" when given some hypergraph H as input.We have to show that then H has an HD of width ≤ k.Algorithm log-k-decomp returns "true" when the call of function Decomp on line 3 with the connection subhypergraph H comp returns "true." Hence, it suffices to show that function Decomp is sound, i.e., if Decomp returns "true" when called with a connection subhypergraph H as parameter, then H has an HD of width ≤ k.Moreover, we have to show that by materialising the decompositions implicitly constructed in the recursive calls of function Decomp, an HD of width ≤ k of H can be constructed in polynomial time whenever Decomp returns "true.It remains to consider the case that "true" is returned on line 35.This means that, for a particular value of λ(p) (chosen on line 9) and of λ(c) (chosen on line 17), all recursive calls of function Decomp (on lines 28 and 33) return "true." By the induction hypothesis, we may assume that for each of the connection subhypergraphs processed by these recursive calls of Decomp, an HD of width ≤ k exists.Note that we are making use of Claim A here in that we may assume that all recursive calls of Decomp are with properly defined connection subhypergraphs (in particular, the vertex set supplied as third component of the parameter is covered by the edges and special edges in the first two components of the parameter of each such call).Now look at these recursive calls: We are studying a call of function Decomp with parameter H , where H is a connection subhypergraph consisting of a set H .E of edges, a set H .Sp of special edges and a set H .Conn of vertices.The current call of function Decomp apparently has chosen labels λ(p) and λ(c) for nodes p and c, such that all checks on lines 11, 15, 19, and 23, are successful in the sense that program execution continues with these values of λ(p) and λ(c).In particular, there exists a [λ(p)]-component comp down of H , satisfying the conditions V (comp down ) ∩ H .Conn ⊆ λ(p) (line 15) and V (comp down ) ∩ λ(p) ⊆ χ (c) (line 19).
Let {x 1 , . . ., x } denote the set of [χ (c)]-components of H inside comp down .Then H .E ∪ H .Sp (i.e., the set of edges and special edges in H ) can be partitioned into the following disjoint subsets: From the first two kinds of sets of edges and special edges, the following connection subhypergraphs are constructed, for which function Decomp is then called recursively on lines 28 and 33: -for each x i consisting of a set of edges x i .E and special edges x i .Sp, define By assumption, the recursive calls of Decomp for each of these connection subhypergraphs return the value "true." Thus, by the induction hypothesis, for each of these connection subhypergraphs, there exists an HD of width ≤ k.From these HDs, we construct an HD of H as follows: -First take the HD of H ↑ .We shall refer to this HD as D ↑ .Let r denote the root node of D ↑ .By Conn ↑ = H .Conn, we have H .Conn ⊆ χ (r ).-Recall that χ (c) was added as a special edge to the connection subhypergraph H ↑ .Hence, by Definition 3.2, the HD D ↑ has a leaf node u with λ(u) = {χ (c)} and χ (u) = χ (c).Now we replace node u in D ↑ by node c with λ(c) and χ (c) according to the current execution of function Decomp.Moreover, for every f ∈ H .Sp with f ⊆ χ (c), we append a fresh child node c f to c with λ(c f ) = { f } and χ (c f ) = f .It is easy to verify that the resulting decomposition (let us call it D ) is an HD of the connection subhypergraph that contains all edges and special edges of H except for the ones in any of the x i 's.In particular, node r with H .Conn ⊆ χ (r ) is still the root of HD D .-Now we take the HDs D i of the connection subhypergraphs (x i .E, x i .Sp, x i .Conn) and append them as subtrees below c in D , i.e., the root nodes of the HDs D i become child nodes of c.Let us refer to the resulting decomposition as D. It remains to show that D indeed is an HD of width ≤ k of the connection subhypergraph H of H .The width is clear, since all HD-fragments of D and also λ(c) have width ≤ k.It is also easy to verify that every edge in H .E is covered by some node in D and every special edge in H .Sp is covered by some leaf node in D.Moreover, also the connectedness condition holds inside each HD-fragment (by the induction hypothesis) and between the various HD-fragments.The latter condition is ensured by the definition of components in Definition 3.6 and by the fact that any two connection subhypergraphs processed by the various recursive calls of function Decomp can only share vertices from χ (c).Finally, note that the above construction of HD D from the HD-fragments constructed in the recursive calls of Decomp is clearly feasible in polynomial time.
Before we prove the completeness of algorithm log-k-decomp, we introduce a special kind of connection subhypergraphs: Let H be a hypergraph, and let D = T , χ, λ be an HD of H with root r .We call H = (E , Sp, Conn) a D-induced connection subhypergraph of H if there exists a subtree T of T with the following properties: -E = cov(T ); -let B denote those nodes in T that are outside T but whose parent node is in T ; then Sp = {χ (u) | u ∈ B}. -If the root r of T has the parent p in T , then Conn = V (H ) ∩ λ(p).Otherwise, if the root r of T is also the root of T , then Conn = ∅.An HD D = S , χ , λ of H is then obtained as follows: -the tree S of D is the subtree of T induced by the nodes of T plus the nodes in B; -for all nodes u in T , we set χ (u) = χ (u) and λ (u) = λ(u); -for all nodes u in B, we set χ (u) = χ (u) and λ (u) = {χ (u)}.We shall refer to D as the induced HD of H .It is easy to verify that D is in normal form, whenever D is in normal form.In the completeness proof below, we shall refer to a D-induced connection subhypergraph of H simply as an "induced subhypergraph" of H .No confusion can arise from this, since we will always consider the same HD D of H throughout the proof.
Completeness Proof.Suppose that hypergraph H has an HD of width ≤ k.We have to show that then algorithm log-k-decomp returns "true." This is the case if the call of function Decomp with parameter H comp on line 3 returns "true, " where H comp is the connection subhypergraph (E(H ), ∅, ∅).We are assuming that H has an HD of width ≤ k.Of course, this is then also an HD of H comp .By Theorem 3. Let us refer to this balanced separator as the node c in D .By the balancedness, it can be easily verified that c must satisfy λ(c) ⊆ E(H ) (that is, c is not a leaf node with λ(c) = { f } for some special edge f ).We distinguish the following three cases.Case 1. Suppose that c is the root node of D and c is different from the root of D. Hence, c has a parent node in D. Let us refer to this parent node as p.If function Decomp has not already returned "true" before, then it will eventually try λ(p) in the foreach-statement on line 9. Due to the normal form of D , all of H is a single [λ(p)]-component.Hence, the if-condition on line 11 is satisfied and comp down is assigned all of H on line 12.The connectedness check on line 15 succeeds, since p is the parent of the root of D and H .Conn = V (H ) ∩ λ(p) holds by the last condition of the definition of induced subhypergraphs.Hence, the foreach-loop on lines 17-35 is eventually entered.If function Decomp does not return "true" before, then it will eventually try λ(c) in the foreach-statement on line 17.Then χ (c) assigned on line 18 is the correct χ -label of c according to the normal form.The connectedness check on line 19 succeeds, since D satisfies the connectedness condition.By assumption, c is a balanced separator; hence also the check on line 23 succeeds.Thus, the foreach-loop on lines 25-29 is executed.It is easy to verify that the parameters supplied to Decomp in the recursive calls on line 28 correspond to induced subhypergraphs.Therefore, all these calls of Decomp return "true" by the induction hypothesis.Hence, also the statements on lines 30-33 are executed.In this case, since comp down comprises all edges and special edges of H , Decomp is called on line 33 with comp up .E = ∅ and comp up .Sp = {χ (c)}.Hence, as was shown in the induction begin, this call of Decomp returns "true." Therefore, the return-statement on line 35 is executed and the overall result "true" is returned by function Decomp.
Case 2. Suppose that c is the root node of D and c is also the root of D. It is easy to verify that, in this case, the call of Decomp with parameter H satisfies H .Conn = ∅.Indeed, the original call of Decomp on line 3 is with H .Conn = ∅.Moreover, all recursive calls of Decomp on line 28 for the HD-fragment "above" node c let the Conn-part (i.e., the third part) of the parameter unchanged.
If function Decomp does not return "true" before, then it will eventually try λ p = ∅ in the foreachstatement on line 9. Then comps p on line 10 contains a single [λ p ]-component, consisting of all edges and special edges in H .E ∪ H .Sp.Hence, the check on line 11 succeeds and this single component will be assigned to comp down on line 12.By the above considerations, we have H .Conn = ∅.Hence, the connectedness check on line 15 trivially succeeds and the foreach-loop on lines 17-35 is eventually entered.If function Decomp does not return "true" before, then it will eventually try λ(c) in the foreach-statement on line 17.Then χ (c) is assigned the correct χ -label of c according to the normal form on line 18.Finally, the connectedness check on line 19 trivially succeeds due to λ c = ∅.
From now on, the argumentation is exactly as in Case 1: By assumption, c is a balanced separator; hence also the check on line 23 succeeds.Thus, the foreach-loop on lines 25-29 is executed.It is easy to verify that the parameters supplied to Decomp in the recursive calls on line 28 correspond to induced subhypergraphs.Therefore, all these calls of Decomp return "true" by the induction hypothesis.Hence, also the statements on lines 30-33 are executed.In this case, since comp down comprises all edges and special edges of H , Decomp is called on line 33 with comp up .E = ∅ and comp up .Sp = {χ (c)}.Hence, as was shown in the induction begin, this call of Decomp returns "true." Therefore, the return-statement on line 35 is executed, and the overall result "true" is returned by function Decomp.
Case 3. Suppose that c is not the root node of D .Then c has a parent node inside D .Let us refer to this parent node as p.If function Decomp has not already returned "true" before, then it will eventually try λ(p) in the foreach-statement on line 9.By Corollary 3.12, the [χ (p)]-component C p consisting of the edges and special edges that are covered in D by the subtree rooted at child node c of p is also a [λ(p)]-component.Hence, one of the [λ(p)]-components computed on line 10 is this [χ (p)]-component C p .The check on line 11 is successful, because the child c of p is a balanced separator.Hence c and the subtrees below c cover more than half of the edges and special edges.The check on line 15 succeeds because of the connectedness condition in D. Hence, the foreach-loop on lines 17-35 is eventually entered.If function Decomp does not return "true" before, then it will eventually try λ(c) in the foreach-statement on line 17.Then χ (c) assigned on line 18 is the correct χ -label of c according to the normal form.The connectedness check on line 19 succeeds, since D satisfies the connectedness condition.By assumption, c is a balanced separator; hence also the check on line 23 succeeds.Thus, the foreach-loop on lines 25-29 is executed.It is easy to verify that the parameters supplied to Decomp in the recursive calls on line 28 correspond to induced subhypergraphs.Hence, all these calls of Decomp return "true" by the induction hypothesis.Hence, also the statements on lines 30-33 are executed.Again, it is easy to verify that also the parameters supplied to Decomp in the recursive call on line 33 correspond to an induced subhypergraph.Hence, by the induction hypothesis, also this call of Decomp returns "true." Therefore, the return-statement on line 35 is executed and the overall result "true" is returned by function Decomp.

AN ILLUSTRATIVE EXAMPLE
To illustrate the notions introduced in Section 3 and the basic algorithm log-k-decomp shown in Algorithm 1, we consider the hypergraph H = (V , E) with V = {x 1 , . . ., x 10 } and In other words, H is a the cycle graph with 10 vertices x 1 , . . ., x 10 .A hypertree decomposition D of H is shown in Figure 5(a).
We now walk through algorithm log-k-decomp, which will allow us to see also the notions from Section 3 in action.Suppose that we run log-k-decomp with hypergraph H and parameter k = 2.
In each of the loops on lines 9 and 17, the algorithm searches for a λ-label until it finds a successful one.By "successful, " we mean that the current execution of the main program or of function Decomp returns "true" (on lines 6, 8, or 35, respectively).To keep things simple in our discussion below, we will directly choose a successful one with the understanding that this particular λ-label will eventually be selected by the program unless another successful one has already been found before.
Main program.The algorithm begins by creating a connection subhypergraph H comp with all the edges {R 1 , . . ., R 10 }, and empty sets for the special edges and for Conn.Then we call the function Decomp with H comp as input (line 3).This component satisfies both the constraint for connectedness (line 19) and the size constraint (line 23).Moreover, it leads to two recursive calls of function Decomp: one for the component comps c [i] = {R 7 , R 8 , R 9 , R 10 } "below" the "child node" (on line 28) and one for the component } "above" (on line 33).For the component "below, " we have x .E = {R 7 , R 8 , R 9 , R 10 } and x .Sp = ∅; moreover, we set x .Conn = {x 1 , x 7 } on line 26.For the component "above, " we set The recursive call of function Decomp on line 33 for the "components above" is a special case where there are actually no such "components above" left.Hence, in this case, we have  and H .Conn = ∅.The execution of this function call is very similar to the calls discussed above.Below, we therefore do not discuss in detail the remaining recursive calls inside Call 1.2.Instead, we only list for each such call the parameters, the balanced separator χ c , and the corresponding HD-fragments.

Call 1.2 of function Decomp with parameters
In Call 1.2 of function Decomp, eventually the balanced separator with λ c = {R 1 , R 3 } and χ c = {x 1 , x 3 , x 4 } will be chosen, which gives rise to the recursive Calls 1.2.1 (line 28) and 1.2.2 (line 33) of function Decomp, which we briefly discuss below.and H .Conn = {x 1 , x 4 }.In Call 1.2.1 of function Decomp, eventually the balanced separator with λ c = {R 1 , R 5 } and χ c = {x 1 , x 5 , x 6 } will be chosen, which in turn gives rise to the recursive Calls 1.2.1.1 (line 28) and 1.2.1.2(line 33) of function Decomp.Both of these calls are successful (as we will show below) and the corresponding HD-fragment D 1.2.1 will combine the HD-fragments from Calls 1.2.1.1 and 1.2.1.2:From Call 1.2.1.2we will have a leaf-node with special edge s 5 , which has the same vertices as χ c .We replace this leaf with a new node with λ set to λ c .Below this new node, we attach the resulting HD-fragment D 1.2.1.1 from the corresponding call, which consists of only a single node with λ-label {s 1 }.Below we briefly discuss how the two Calls 1.2.1.1 and 1.2.1.2are processed.

Call 1.2.1 of function Decomp with parameters
Call 1.2.1.1 of function Decomp with parameters H .E = ∅, H .Sp = {s 1 } with s 1 = {x 1 , x 6 , x 7 }, and H .Conn = {x 1 , x 6 }.We now have a connection subhypergraph of H consisting of only a single special edge.Thus we hit the base case on line 8 and return "true".The corresponding HD-fragment D 1.2.1.1 consists of one node: the node with λ-label {s 1 }.Call 1.2.1.2 of function Decomp with parameters H .E = {R 4 }, H .Sp = {s 5 } with s 5 = {x 1 , x 5 , x 6 }, and H .Conn = {x 1 , x 4 }.As in the Call 1.1.2,we now have a connection subhypergraph of H consisting of a single edge and a single special edge.Analogously to Call 1.1.2,also Call 1.2.1.2returns "true" and the corresponding HD-fragment D 1.2.1.2consists of 2 nodes: the node with λ-label {R 1 , R 4 } and its child node with λ-label {s 5 }. and H .Conn = ∅.Analogously to the Call 1.2.1, we have a connection subhypergraph of H consisting of two edges and a single special edge, and again this call of Decomp will return "true".Since the recursive calls of Call 1.2.2 are nearly identical to Call 1.2.1, we omit mentioning them explicitly.The corresponding HD-fragment D 1.2.2 consists of two nodes: the root node with λ-label {R 1 , R 2 } and its child node with λ-label {s 4 }.Note that the root of this fragment also happens to be the root of D itself, as shown in Figure 5(a).The reason for this lies in the fact that Call 1.2.2. is reached by always following the recursive call in line 33, where we determine the HD-fragment for the component above the child.As these HD-fragments make up the HD we are trying to construct, we are thus traversing this HD upwards and thus eventually reach the root node of D.

Call 1.2.2 of function Decomp with parameters
We can now construct the HD-fragment D 1.2 of the successful Call 1.2 by taking HD-fragment D 1.2.2 , replacing the leaf node with λ-label {s 4 } by the node c with λ c = {R 1 , R 3 } and χ c = {x 1 , x 3 , x 4 } from Call 1.2 and appending the HD-fragment D 1.2.1 below this node c.The resulting HD-fragment D 1.2 is shown in Figure 5(c).

FURTHER COMBINATORIAL OBSERVATIONS AND OPTIMISATIONS
As shown in Theorem 4.1, algorithm log-k-decomp introduced in Section 4 reaches the primary goal of splitting the HD-construction into subtasks with guaranteed upper bound on the size of the subtasks.In theory, this is enough to support parallelism.However, this basic algorithm still leaves a lot of room for further improvements.In this section, we present several optimisations, which are crucial to achieve good performance in practice.The line numbers below refer to Algorithm 1.However, in Algorithm 2, we will ultimately also give the pseudo-code for the enhanced algorithm where all the optimisations mentioned below are included.
Extension of the base case.The recursive function Decomp starts (on lines 5-8) with some simple checks that immediately give a "true" answer.In contrast, a "false" answer is only obtained in case of unsuccessful execution of the entire procedure.We could add the following negative case to the top of the procedure: If H .E = ∅, then |H .Sp| ≤ 1 must hold.The rationale of this condition is that if there are no more edges in H .E, then we would have to use only "old" edges (i.e., edges covered already at some node further up in the HD) in the λ-label to separate the remaining special edges.However, a λ-label consisting of "old" edges only is not allowed, since this would violate the second condition of the normal form in Definition 3.8.

Root of the HD-fragment.
In the current form of procedure Decomp, we always "guess" a pair (p, c) of nodes, such that p is the parent of c.This also covers the case that c is the root node of the HD-fragment for the current connection subhypergraph.In this case, the parent node p would actually be the node immediately above this HD-fragment (in other words, p was the node from which the current call of Decomp happened) or a "dummy" node (if the balanced separator is the root of the HD of the input hypergraph H and, therefore, does not have a parent node).However, it would be more efficient to consider the case of "guessing" the root node of this HD-fragment explicitly.More precisely, we would thus first check for the label λ p guessed in Decomp on line 9 (which, in the current version of the algorithm, is automatically treated as the "parent") if all [λ p ]components have at most half the size of the current connection subhypergraph.
If this is the case, then we may use this node as the root of the HD-fragment to cover the current connection subhypergraph.This makes sense, since it corresponds precisely to the "search" for a balanced separator in the proof of Lemma 3.14.That is, if the root of the HD gives rise to components that are all at most half the size, then this may actually be the desired balanced separator.
If this is not the case, then we simply proceed with procedure Decomp in its present form, i.e, there exists exactly one [λ p ]-component whose size is bigger than half.So we take the guessed node as the parent and search for a balanced separator as a child of p in the direction of this oversized [λ p ]-component.
Allowed edges.The main task of procedure Decomp is to compute labels (i.e., edge sets) λ p and λ c of nodes p, c, which will ultimately be in a parent-child relationship in the HD.For these labels, Algorithm 1 imposes no restriction.That is, in principle, we would try all possible sets of ≤ k edges for these labels.However, not all edges actually make sense.We should thus add one more parameter to procedure Decomp indicating the edges that are allowed in a λ-label of the HD-fragment for this connection subhypergraph.
More specifically, in our search for the λ-label of some node u, we may exclude from the HD of the connection subhypergraph H comp up (i.e., in the recursive call of function Decomp on line 33) all edges which are part of some component "below" u.The rationale of this restriction is that, by the special condition, using a "new" edge in a λ-label forces us to add all its vertices to the χ -label, i.e., it is fully covered in such a node.But then it cannot be part of a component whose edges are covered for the first time further down in the tree.
Note that we can yet further restrict the search for the label λ c by requiring that at least one edge must be from H .E, since choosing only "old" edges would violate the second condition of the normal form.As far as the label λ p is concerned, the same kind of restriction can be applied if we first implement the previous optimisation of handling the root node of the current HD-fragment separately.If we indeed have to guess the labels λ p and λ c of two nodes p and c (i.e., the label λ p guessed first was not a balanced separator), then both nodes p and c are inside the current HD-fragment.Hence, also the label λ p must contain at least one "new" edge.
Searching for child nodes first.In Algorithm 1, we first look for λ-labels of potential parent nodes, and consider afterwards the λ-labels of potential child nodes.Only then do we check if the χ -label of the child is a balanced separator of the current subcomponent.We have observed that in many hypergraphs of HyperBench, balanced separators are rare, in the sense that only a small part of the search space will ever fulfil the properties required.Therefore we should first look for a potential child s.t.its λ-label is a balanced separator, and only afterwards try to find a fitting parent.While this may seem slightly unintuitive, it allows us to quickly detect cases where no balanced separator can be found at all.
In principle, we can determine the precise bag χ c for a child c only when we know the λ-label of its parent.Nevertheless, even if we only have λ c , we can over-approximate the χ c -label as λ c .Hence, if λ c is not a balanced separator, then we may clearly conclude that neither is χ c .Actually, the "over-approximation" is not significant anyway: When we later determine an appropriate λ-label of the parent p of c, we will compute the "downward"-component C p whose 1:31 size is bigger than . The HD-fragment rooted at c is then supposed to cover all of C p .By Corollary 3.12, we know that the "downward"-component C p is both, a [λ(p)]-component and a [χ (p)]-component.Hence, by Lemma 3.11, we know that, inside C p , there is no difference between the [χ (c)]-components and the [λ(c)]-components.
Finally, note that by searching for the child node first, we get the above described optimisation of treating the "Root of the HD-fragment" separately almost for free.Indeed, when computing λ c , we can immediately check if H .Conn ⊆ λ c holds.Recall that H .Conn constitutes the interface to the HD-fragment above the current one.Hence, if λ c fully covers this interface, then c may be the root node of the current HD-fragment, which is checked on lines 15-22 in Algorithm 2. Of course, even if λ c is a balanced separator, this does not necessarily mean that it is indeed the root of the current HD-fragment.Hence, either the check on line 15 in Algorithm 2 or one of the recursive calls on line 20 may fail.In this case, we would simply proceed to the search for the label λ p of the parent node p of c.Note that we can now be sure that p is inside the HD-fragment of H . Hence, in contrast to Algorithm 1, the case of λ p = ∅ does not need to be considered here, i.e., on line 23 in Algorithm 2, we have the condition 1 ≤ |λ p | ≤ k rather than 0 ≤ |λ p | ≤ k as was the case for the "ParentLoop" in Algorithm 1.
Speeding up the search for parent λ-labels.The previous optimisation means that, after having found a λ-label λ c for the child that is a balanced separator of the current subcomponent, we need to find a suitable λ-label of the parent.By "suitable" we mean that we may limit ourselves to edges that have a non-empty intersection width λ c .A very high-level explanation why we may exclude edges e with e ∩ λ c = ∅ from the search space of λ p is that the control flow of function Decomp is mainly determined by the edges and special edges covered by λ c and the [λ c ]-components below c.By the connectedness condition, if e is covered above c and has empty intersection width λ c , then excluding or including e in λ p has no effect on the [λ c ]-components below c.In our experimental evaluation, we found that this restriction indeed significantly reduces the time it takes to either find a suitable λ p , or detect that no such λ-label exists.Of course, this restriction of the search space cannot destroy soundness.We will show below that also the completeness of the algorithm is preserved.
Theorem 7.1.The optimised log-k-decomp algorithm for checking if a hypergraph H has hw(H ) ≤ k given in Algorithm 2 is sound and complete.More specifically, for given hypergraph H and integer k ≥ 1, the algorithm returns "true" if and only if there exists an HD of H of width ≤ k.
Proof.The soundness and completeness of Algorithm 2 follow almost immediately from the soundness and completeness of Algorithm 1 together with the above explanations of the various optimisations.The only non-trivial part is that the last optimisation (i.e., the restriction of the search space for λ(p)) does not destroy the completeness of the algorithm.The remainder of the proof will concentrate on this aspect.
Assume that hypergraph H has an HD of width ≤ k.Then the optimised log-k-decomp algorithm without the restriction on the search space for label λ p (on line 23) returns the overall result true.This is due to the fact that, as was argued above, the other optimisations mentioned in this section do not affect the completeness of the algorithm.Now consider a recursive call of function Decomp and suppose that it returns true if the restriction on the search space for label λ p is dropped.Of course, if the value true is returned in one of the base cases (lines 6 or 8) or if λ c turns out to be the λ-label of the root node of the current HD-fragment (and true is returned on line 22), then the restriction of the search space for λ p has no effect at all.Hence, the only interesting case to consider is that the "ParentLoop" (lines 23-47) is indeed executed.Let λ p be the λ-label chosen on line 23 if no restriction is imposed on the search space.We claim that we may remove from λ p all edges that have an empty intersection width λ c without altering the control flow of this particular execution of function Decomp.Actually, it suffices to show that we may remove one edge e with an empty intersection width λ c from λ p without altering the control flow of this particular execution of function Decomp.Then the claim follows by an easy induction argument.
So suppose that λ p contains at least one edge e such that e ∩ λ c = ∅, and let λ p = λ p \ {e}.An inspection of the code of the "ParentLoop" reveals that it suffices to show that this elimination of edge e from λ p leaves comp down unchanged.Indeed, if comp down is still a [λ p ]-component, say, the ith [λ p ]-component, then the if-condition on line 25 is true.Of course, there can be only one . Hence, on line 26, for this particular i, exactly the same value is assigned to comp down for λ p as for λ p .But then also χ c on line 29 gets the same value as without the restriction on the search space of λ p .Consequently, also the By comp down ⊆ comp down , all [λ c ]-components C 1 , . . .,C contained in comp down are of course also contained in comp down .We have to show that there is no further [λ c ]-component contained in comp down .Assume to the contrary that there exists a [λ c ]-component C in comp down such that C is not in comp down .By definition, comp down is [λ p ]connected while comp down is [λ p ]connected.Hence, there exist (possibly special) edges f ∈ C and f ∈ C i for some i ∈ {1, . . ., }, such that there is a path π (represented as a sequence of edges) with π = (f 0 , f 1 , . . ., f m ), such that f = f 0 , f = f m , and (f α ∩ f α +1 ) \ λ p ∅ for every α ∈ {0, . . .,m − 1}.W.l.o.g., choose f , f , and π such that m is minimal.Since f and f are not [λ p ]-connected, there exists α with Since all (special) edges in comp down are either in some [λ c ]-component contained in comp down or covered by λ c , and since we are assuming that π is of minimal length, the path π starts with f in some [λ c ]-component C i , possibly goes through λ c and ends with f in component C .Recall that e was chosen such that e ∩ λ c = ∅.Hence, the edges f α and f α +1 cannot be covered by λ c .By our assumption that π has minimal length, we can also exclude the case that both f α and f α +1 are in C .Hence, at least one of f α and f α +1 must be in C i .In other words, e ∩ C i ∅.Hence, also e ∩ comp down ∅.However, by the check on line 32 in Algorithm 2, we know that V (comp down ) ∩ λ p ⊆ λ c .This contradicts the assumption that e ∈ λ p and e ∩ λ c = ∅.

IMPLEMENTATION AND EVALUATION
We report now on the empirical results obtained for our implementation of the log-k-decomp algorithm.Our experiments are based on the HyperBench benchmark from [12], which was already used for the evaluation of previous decomposition algorithms, notably NewDetKDecomp [12] (an enhanced re-implementation of det-k-decomp [27]) and HtdLEO [31].
The goal of the experiments is to determine the exact hypertree width of as many instances as possible.We compare here the performance of three different decomposition methods, namely NewDetKDecomp [12], HtdLEO [31], and our implementation of log-k-decomp.Note that while the tested implementations include the capability to compute GHDs or FHDs, we only consider the computation of HDs in our experiments here.Our new implementation of log-k-decomp is based on the open-source code of BalancedGo [26], a parallel algorithm for computing GHDs.
Parallel Implementation.For our experiments, we implemented log-k-decomp including all of the optimisations presented in Section 7. As discussed above, a crucial aspect of our algorithm design is that the use of balanced separators allows us to recursively split the problem into smaller subproblems.The subproblems are independent of each other and are therefore processed in parallel by our implementation.Furthermore, following observations made in Reference [26], our implementation also executes the search for balanced separators in parallel by partitioning the search space effectively.
The full raw data of our experiments is publicly available [18], as is the source code of our implementation1 of log-k-decomp.

Benchmark Instances and Setting
For the evaluation, we use the benchmark library HyperBench [12].It contains 3,648 hypergraphs underlying CQs and CSPs from various sources in industry and the literature and is commonly used to evaluate decomposition algorithms.
A number of our experiments are performed over the full set of HyperBench instances.However, for some of our experiments it is more meaningful to restrict them to exclude hypergraphs that are, roughly speaking, too small or have high width.Small instances benefit only marginally from algorithmic improvements or parallelism, while very high width is of less algorithmic interest as it exponentially effects algorithms that make use of decompositions.Hence, we propose to exclude such instances in some cases to make more relevant observations.For the purpose of creating this restricted set of instances, we focus on instances with more than 50 edges and vertices that are known to have hypertree width at most 6.There are 465 instances in HyperBench that satisfy these conditions; we will refer to them throughout this section as HB larдe .
The instances are available at http://hyperbench.dbai.tuwien.ac.at and also in the published raw data of our experiments [18].
Hardware and Software.Our implementation is written in the programming language Go using version 1.14, and we will refer to it as log-k-decomp.We will give more details below on how it is configured for the experiments we report in Section 8.4.The hardware we use for the evaluation is a cluster of 12 nodes, using Ubuntu 16.04.1 LTS, with Linux kernel 4.4.0-184-generic,GCC version 5.4.0.Each node has a 12 core Intel Xeon CPU E5-2650 v4, clocked at 2.20 GHz and using 264 GB of RAM.

Setup of Experiments.
For the current extended version of this work, we used the following time and memory limits: a timeout of one hour is used uniformly for all systems and we limited available RAM to 1 GB for NewDetKDecomp, to 24 GB of RAM for HtdLEO, and to 40 GB of RAM for log-k-decomp when using the hybrid version (to be discussed in detail in Section 8.3).The differences are explained by choosing for each system a limit such that no "out of memory" error was reported on our machines and it thus roughly reflects an upper bound on the needed amount of memory for each system to run every single instance in our used benchmark.For log-k-decomp, each run needs two inputs: a hypergraph H and the width parameter k ≥ 1.For these tests, we use width parameters in the range [1,10].A peculiarity of HtdLEO is that it needs no width parameter, since it directly tries to find an optimal solution.We use the HTCondor system [32] to execute the tests.HTCondor is also used to manage the limits to memory and number of cores accessible by the test instances.
Throughout this section, we will be interested in two key metrics.First, the number of solved instances, by which we mean instances for which an optimal (i.e., minimal width) hypertree decomposition is found and proven optimal.Second, the computation time that is necessary to compute the optimal width decomposition, which we will refer to as the runtime.The "optimal (hypertree-)width" is always understood as the known optimal width, using all sources of information available to us: the HyperBench website, past published experiments, and our own experiments.
Importantly, this means that average runtimes are taken only over the instances that the respective algorithm is able to solve, while timed out instances are not considered in the runtime calculation.

Implementation Improvements
Parts of this article were published earlier in Reference [17] where we reported on preliminary versions of some of the experiments reported here.We have since improved the implementation of the algorithm in various critical aspects and in consequence we wish to note that the results in this article show significant improvement over those reported in Reference [17].Here we briefly summarise these changes.
Detailed low-level analysis of the previous implementation revealed that a significant portion of the time was spent on memory management in the Go running time.Go uses a garbage collector to automatically manage dynamically allocated memory and both frequent allocation of memory and garbage collection itself made up a substantial percentage of the overall execution time.Critically, the proportion of time spent in memory management increased as the number of parallel execution threads increased.We observed that the number of memory allocations scaled linearly with the number of parallel threads, thus also triggering the garbage collection more frequently in proportion to the number of threads.Our analysis showed that the ever-increasing time spent with memory management was a key factor of the diminishing returns that we observed when scaling our implementation to more than four CPUs in the preceding conference paper [17].
To alleviate the high costs of memory management, we improve our implementation in two main aspects.First, through various low-level optimisations we greatly reduce the number of memory allocations performed when computing the [S]-components for a separator S.This is the key subroutine in the check for balanced separation (cf.Algorithm 7.1) and our changes reduce the number of overall memory allocations by orders of magnitude.In addition, we configure the garbage collector trigger to adapt to the proportion to the number of CPUs used.That is, when using the n CPU cores, we set the garbage collector trigger condition to be n times less sensitive than the standard setting, thus approximately achieving the same overall frequency of garbage collection for any number of CPUs.
These two measures together, significantly reducing memory allocations overall as well as throttling the garbage collector so that it does not dominate the execution time as the number of used CPUs increases, help us achieve both better performance overall-we manage to solve more than Fast Parallel Hypertree Decompositions 1:35 150 new instances optimally within the timeout of one hour and crucially also improve scaling behaviour when using more CPUs.

Hybrid Approaches
While our algorithm has desirable properties for parallelisation (in particular, the logarithmically bounded recursion depth established in Theorem 4.1), this comes at the cost of some overhead when compared to simpler methods, in particular det-k-decomp.Especially on small and simple instances, the restriction to balanced separators may act as a detriment to performance that outweighs its benefits for parallelisation and its effect on severely restricting the search space.
To balance these considerations overall in practice, we therefore also consider hybrid variants of our implementation.Intuitively, we want to use log-k-decomp as long as the subproblems are still complex, but once they become simple, we want to switch to an algorithm that is better suited for those cases.For the simpler algorithm, det-k-decomp is the natural choice, as it performs very well on small instances as is shown in Reference [12], where an implementation of det-k-decomp is provided as part of NewDetKDecomp.To determine when the switch is made, we implement two simple metrics to capture the complexity of a hypergraph: where k is the width parameter of the algorithm.The additional factor compared to EdgeCount is best understood as two separate additional weightings.Higher width implies more complex structure and hence we expect more complexity per edge.However, if edges are on average larger, then it becomes easier to find covers and we therefore also inversely weight by the average cardinality of the edges.
We investigate the effectiveness of these metrics through a series of experiments.In particular, we conduct experiments with both metrics and different thresholds for when to switch from log-k-decomp to det-k-decomp.To be precise, for a metric m and threshold T , log-k-decomp is executed for a subproblem with hypergraph H i as long as m(H i ) ≥ T .If m(H i ) < T , then we switch to an implementation of det-k-decomp written from scratch as part of the code base of log-k-decomp.A similar strategy is also proposed in Reference [26] in the context of BalancedGo.However, in that system no metric for the complexity of a subproblem is employed, but rather the switch to det-k-decomp is always performed at a fixed recursion depth.
Our results for the experiments on HB larдe are summarised in Table 1.Methods WeightedCount and EdgeCount refer to log-k-decomp with the respective metric used for hybridisation.The threshold column refers to parameter T in the discussion above.The experiments for all log-k-decomp hybrid methods had access to 12 cores, the experiments for NewDetKDecomp and HtdLEO used only 1 core each, since they do not support parallelism.
Overall, WeightedCount clearly performs best, especially in the number of solved instances.For thresholds 400 and 600, approximately 90% of the 465 large instances from HB larдe are solved.This constitutes a significant improvement over the 37% and 60% achieved by NewDetKDecomp and HtdLEO, respectively.Note that despite solving more instances-for which det-k-decomp and HtdLEO timed out-the runtime is also at least 3 times lower for WeightedCount.This is surprising, as we do not consider timed-out instances in our average runtime calculations.
One surprising observation from the table is that the differences in performance between different thresholds are much smaller for WeightedCount than for EdgeCount.Further investigation suggests that this is due to the WeightedCount metric decreasing much more rapidly as hypergraphs become simpler.At the same time, for subproblems that fall in the range between 200 and 600, the performance of switching to det-k-decomp immediately is roughly the same (on average) as the performance of continuing with log-k-decomp for one or two more steps.While EdgeCount performs worse than WeightedCount, we can still see a clear improvement over the state-of-the-art methods NewDetKDecomp and HtdLEO.Especially the significant improvement over det-k-decomp is important to observe as it clearly demonstrates the benefits of our hybrid approach.Recall that when we split our problem into balanced subproblems, each subproblem is then solved independently in parallel.In the hybrid variant, we will thus eventually execute our implementation of the det-k-decomp algorithm on multiple subproblems in parallel, i.e., we can use an inherently single-threaded algorithm effectively in parallel, because we are able to create balanced subproblems.

Empirical Evaluation
We compare the aforementioned hybrid version of the log-k-decomp algorithm with the two state-of-the-art implementations for finding HDs: NewDetKDecomp [12] and HtdLEO [31].
Our results are summarised in Table 2, distinguishing the hypergraphs in the HyperBench benchmark by size and origin. 2 We distinguish between two main categories, hypergraphs that are derived from applications and hypergraphs that were synthetically generated.In each group, we report our results split by the number of edges |E| in the instance.Note that the group |E| > 100 of instances with more than 100 edges is empty for the Application case and thus omitted from the table.Instances in Group reports the number of instances in each such group.For each algorithm and each group of instances; we list the number of solved instances (#solved) and statistics over the runtimes (avg, max, stdev).Times are all in seconds and rounded to a single digit after the comma.Results over all groups are given in the last row titled "Total." As mentioned above, some care is required when comparing times between algorithms.While NewDetKDecomp has low average time overall, this is partly due to solving fewer instances.The data therefore demonstrate that, in general, NewDetKDecomp either solves an instance quickly or fails to find an optimal width decomposition before timing out.Overall, we see that despite solving significantly more instances than its competitors, runtimes for log-k-decomp overall are comparable with NewDetKDecomp and noticeably lower than for HtdLEO.
Fast Parallel Hypertree Decompositions 1:37 Bolded entries in the solved column represent the highest value achieved among the three compared systems for a given group, thus marking the best achieved result.
HtdLEO with 10 Hour Timeout.The method that is used in HtdLEO fundamentally differs from the search algorithm here.In HtdLEO, the problem is encoded to the SAT modulo theories (SMT) setting and the final solving step is handed off to standard SMT solvers.The encoding in HtdLEO is constructed in such a way that no width parameter is handed to the solver; rather, the encoding will always try to return the optimal width as its solution.This is a significant difference to the parameterised search implemented in log-k-decomp (but also previous algorithms such as det-k-decomp and BalancedGo).This difference makes it naturally difficult to compare runtimes and the number of optimal solutions directly.To provide a fuller picture we add here Table 3.In the table, we present the results from running the same experiments with HtdLEO but with the timeout increased to 10 hours.This increase in timeouts naturally makes the average runtimes difficult to compare to the values in Table 2.However, importantly, we see that the increase in solved instances is also moderate, and overall log-k-decomp still solves significantly more instances than HtdLEO with 10 hours of maximum runtime.
It may be of further interest how these numbers compare to the performance of state-ofthe-art algorithms for finding generalised hypertree decompositions.The results reported for BalancedGo [26] (on a comparable system, according to the description of the experiments) show that the best method there solves only 2,924 instances optimally for ghw.In contrast, when considering the same set of instances, log-k-decomp manages to solve 3,250 of the instances  tested there optimally.When looking at the average running times over all instances, we get 25.76 s for BalancedGo, but 20.6 s for log-k-decomp.Thus, it solves considerably more instances and does so faster on average.Furthermore, in none of the cases where BalancedGo finds the optimal ghw is it lower than the optimal hw.In other words, in practice, the additional complexity of GHDs compared with HDs is not compensated by achieving lower width (even if, in theory, no better upper bound on the hw than hw ≤ 3 • ghw + 1 is known [2]).
In our experiments, we also observe that for low widths-i.e., cases where using HDs is most promising in practice-log-k-decomp is essentially able to solve all instances.In particular, of the 2,947 instances with width at most 6, log-k-decomp solves 2,941 (99.8%) instances.In contrast, NewDetKDecomp and HtdLEO time out on 488 and 919, respectively, of those instances.This suggests that log-k-decomp can be a solid foundation for the integration of HDs in practice going forward.
To gain further insight into the performance of each algorithm with respect to solving instances optimally, we investigate the size of solved and unsolved instances.To this end, Figure 6 provides (logarithmic) scatter plots for each of the three algorithms in our tests.In each plot, each instance is positioned according to its number of vertices and edges.Solved instances for each algorithm are drawn in green while unsolved instances are drawn in red.
The plots show that our intuition holds true in that solving large instances (in both axis) significantly benefits from using log-k-decomp.Most of the remaining hypergraphs are either extremely large, containing thousands of edges and vertices, or belong to very specific CSP classes that we Fast Parallel Hypertree Decompositions 1:39 know to have very high width (significantly beyond the width 10 limit used in our experiments) through graph-theoretic arguments.We want to analyse for how many instances of HyperBench each decomposition method could determine its width and specifically focus on how well it fares as the width increases.Note that HtdLEO is unique in this respect, since it determines the optimal width right away.Thus, if we ask for how many instances HtdLEO could determine if its width is ≤5, for example, then we are really just counting how many timeouts there were in general.For the parametrised decomposition methods, however, this question does give us new insights into how its run timescales when looking for decompositions of larger or smaller width.
For this purpose, we first need the concept of the "Virtual Best" method.This notion simply aggregates the results of all considered methods and shows for how many instances of HyperBench we know their hw.We can see in Table 4 how each of the three methods fares when compared against this virtual best method.For widths up to 7, the Hybrid log-k-decomp is unbeaten, solving all known instances, as well as solving many of them exclusively.
To provide a more detailed analysis, we also compare for how many instances of hw up to 6, each method can determine whether an instance has hw of lower than the given number by finding an HD of such a width or determining that no such HD can exist.Note that this does not require proving optimality.We can see the results in Table 5.We can see that both log-k-decomp and log-k-decomp (Hybrid) are very good at this, with the Hybrid determining for 3,437 (over 94% of) instances whether they have hw ≤ 6.If we limit ourselves to hw ≤ 5, then it determines the question for 3,612 or 99% of instances.
We perform another set of experiments over the instances in HB larдe to verify our claims that log-k-decomp is well suited for parallelisation.For 1 ≤ n ≤ 12, we observe the time taken to find and verify the optimal width of an instance using n CPU cores.We report on the times to find these optimum widths averaged over all instances in HB larдe in Figure 7.We will briefly note that the ideal scaling function one could expect here is 1  x , where we normalise the runtime at the sequential (1 core) case as 1, and divide by the number of cores x.To make the comparison against this ideal function easier, we added it to Figure 7 as a dotted line.Another way to illustrate this ideal scaling function is by taking the time when using x cores, and multiplying it with x.This should produce a constant function in the ideal case.We have also added the runtimes multiplied by number of cores to Figure 7.We observe near optimal speedups (in the sense described above)  up to 8 cores, from about 235 s on 1 core to 31 s for 4 cores for log-k-decomp.This behaviour is expected, since our parallelisation strategy relies on dividing up the search space for bounded separators uniformly over the the available cores.Since this requires no communication between 1:41 threads or other overhead that depends on the degree of parallelisation, the key task of searching for balanced separators scales evenly in the number of cores.In instances where the search for separators dominates the runtime, such as negative instances where the full search space is explored, analysis of our algorithm therefore predicts effectively ideal scaling of performance.In the data from Figure 7, we observe diminishing returns in the rate of improvement of average runtime starting from 8 cores, though we still see that the average runtime is reduced.Very similar scaling can be observed for our Hybrid version.

CONCLUSION AND OUTLOOK
In this article, we have introduced a novel algorithm, log-k-decomp, for computing hypertree decompositions.Based on new theoretical insights and results on HDs, we were able to propose an algorithm that constructs decompositions in arbitrary order (rather than, e.g., in a strict top-down manner) while achieving a balanced separation into subproblems.In this way, we have obtained a logarithmic bound on the recursion depth of our algorithm, making it particularly well suited for parallelisation.We evaluated an implementation of log-k-decomp through experimental comparison with the state of the art.On the standard benchmark for hypertree decomposition, we are able to achieve clear improvements both in the number of solved instances and in the time required to solve them.In combination, our theoretical results and experiments demonstrate that log-k-decomp achieves our goal of effective parallel HD computation.We believe that the performance improvements, especially on hypergraphs with low width and for large hypergraphs, lay a strong foundation for more widespread adoption of hypertree decompositions in practice, e.g., for complex query execution in high-performance database applications.
With HD computation for large and complex hypergraphs becoming practically feasible, one of the key challenges that block the use of HDs is quickly becoming less problematic.We therefore consider full integration of hypertree decompositions into existing database systems and constraint solvers to be a natural next step in this line of research.
Experiments suggest that there is significant potential in the study of metrics for hybrid approaches.In particular, how can we decide effectively when to switch from the balanced separation of log-k-decomp to the greedy heuristic-guided method underlying det-k-decomp.This motivates a more in-depth study of hybridisation metrics in the future.

Fig. 2 .
Fig. 2.An example connection subhypergraph of the hypergraph from Figure 1.The vertices are represented by letters.Special edges are marked by green and dotted hyperedges, and the vertices in Conn are marked in bold, red font.Next to it is an HD of width 2 of this connection subhypergraph.

Fig. 3 .
Fig.3.Connected components and their respective separator, visually marked.Definition 3.6 (Connectedness, Components).Let H be a hypergraph, let U ⊆ V (H ) be a set of vertices, and let H = E , Sp, Conn be a connection subhypergraph of H .-We define [U ]-adjacency as a binary relation on E ∪Sp such that two (possibly special) edgesf 1 , f 2 ∈ E ∪ Sp are [U ]-adjacent if (f 1 ∩ f 2 ) \ U ∅ holds.-We define [U ]-connectedness as the transitive closure of the [U ]-adjacency relation.-A [U ]-component of H is a maximally [U ]-connected subset C ⊆ E ∪ Sp.

Condition 3 .
Suppose that (p, c) is a pair of nodes in D that satisfies Condition 1 and 2 but violates Condition 3 of the normal form.Let C p denote the [χ (p)]-component with cov(T c ) = C p .By Condition 1 of HDs in Definition 3.2, we have χ (c) ⊆ λ(c) .Moreover, by the connectedness condition of HDs (Condition 3 of Definition 3.2), we must have χ (p) ∩ Proof.It is convenient to prove the following claim first: Claim A. Let H be a connection subhypergraph of some hypergraph H and let D = T , χ , λ be an HD in normal form of H .Moreover, let p, c be nodes in T such that p is the parent of c and C p is the [χ (p)]-component of H with C p = cov(T c ).Then, for any two (possibly special) edges e, e ∈ C p , the following equivalence holds: e and e are [χ (c)]-adjacent if and only if e and e are [λ(c)]-adjacent.Proof of Claim A. By Condition 1 of Definition 3.2, we have χ (c) ⊆ λ(c).Hence, [λ(c)]adjacency clearly implies [χ (c)]-adjacent.We only need to prove the opposite implication.So suppose e, e ∈ C p are [χ (c)]-adjacent.That is, they share some vertex x with x χ (c).We show that e, e are [λ(c)]-adjacent by proving that x λ(c).By the special condition of HDs (Condition 4 of Definition 3.2), we have χ (T c ) ∩ λ(c) ⊆ χ (c).Moreover, C p = cov(T c ) and, therefore, C p ⊆ χ (T c ). Taking these two subset-relationships together, we thus get C p ∩ λ(c) ⊆ χ (c).Therefore, if x ∈ λ(c) were the case, then, together with x ∈ e ∈ C p , we would have x ∈ χ (c).But x was chosen with the property x χ (c).Hence, also x λ(c) holds.Proof of the "if" direction.Suppose that C c is a [χ (c)]-component of H .By Claim A, if two (possibly special) edges e, e ∈ C p are [χ (c)]-adjacent, then they are also [λ(c)]-adjacent.Hence, any The edges and special edges covered by χ (c) are done and need no further consideration.The edges and special edges in the second and third category are taken care of by recursive calls to the function Decomp (lines 28 and 33).To this end, we compute all [χ (c)]-components C 1 , . . .,C m (line 22).Now suppose that C 1 , . . .,C with 1 ≤ ≤ m are the [χ (c)]-components inside the [λ(p)]component C p .Then the function Decomp is called recursively for each of the [χ (c)]-components C 1 , . . .C (line 28).

Theorem 4 . 1 .
Algorithm log-k-decomp correctly checks for given hypergraph H and integer k ≥ 1 if hw(H ) ≤ k holds.The algorithm is realised by a main program and the recursive function Decomp, whose recursion depth is bounded logarithmically in the number of edges of H , i.e., O(log(|H |)).Moreover, by materialising the decompositions implicitly constructed in the recursive calls of the Decomp function, an HD of H of width ≤ k can be constructed in polynomial time in case of a successful computation (i.e., return-value "true").
" The proof is by induction on |H .E| + |H .Sp|.induction begin.Suppose that |H .E| + |H .Sp| = 1 and that function Decomp returns "true." Hence, we either have |H .E| = 1 and |H .Sp| = 0 or we have |H .E| = 0 and |H .Sp| = 1.In either case, an HD of this connection subhypergraph can be obtained with a single node u by setting λ(u) = { f } and χ (u) = f , where f is the only (special) edge in H .E ∪ H .Sp.This decomposition 1:22 G. Gottlob et al.clearly satisfies all conditions of an HD according to Definition 3.
10, H comp also has an HD D = T , χ , λ of width ≤ k in normal form.Moreover, H comp is clearly a D-induced connection subhypergraph of H . Hence, it suffices to show that function Decomp is complete on induced subhypergraphs of H .That is, if Decomp is called with an induced subhypergraph H that has an HD of width ≤ k, then Decomp returns "true." We proceed by induction on |H .E| + |H .Sp|.induction begin.Suppose that |H .E|+|H .Sp| = 1.That is, we either have |H .E| = 1 and |H .Sp| = 0 or we have |H .E| = 0 and |H .Sp| = 1.In the first case, "true" is returned via the statement on line 6; in the second case, "true" is returned via the statement on line 8. induction step.Now suppose that |H .E| + |H .Sp| > 1.If |H .E| ≤ k and |H .Sp| = 0, then the return-statement on line 6 is executed and the function returns "true." It remains to consider the case that |H .E| > k or |H .Sp| > 1 holds.By Lemma 3.14, the HD D induced by H has a balanced separator.

Fig. 5 .
Fig. 5. Visualisations of the HD constructed as part of Section 6 and the HD-fragments used for its construction.

R 5 }
on line 30 and H comp up .Sp = {s 1 } with s 1 = χ c = {x 1 , x 6 , x 7 } on line 31; moreover, we retain H comp up .Conn = ∅ from the current call of Decomp.Clearly, edge R 6 from H is already covered by χ c = {x 1 , x 6 , x 7 } and does not need to be further considered.As we shall work out next, Call 1.1 of function Decomp for the component "below" will return "true" based on the HD-fragment D 1.1 shown in Figure5(b).Likewise, Call 1.2 of function Decomp for the component "above" will return "true" based on the HD-fragment D 1.2 shown in Figure5(c).The leaf node of D 1.2 contains the special edge s 1 , which acts as a placeholder for the node c with labels λ c = {R 1 , R 6 } and χ c = {x 1 , x 6 , x 7 } from the current call of Decomp.The HD-fragment D 1 of the successful Call 1 of function Decomp is then obtained by taking HD-fragment D 1.2 , replacing the leaf node with λ-label {s 1 } by the node c with λ c = {R 1 , R 6 } and χ c = {x 1 , x 6 , x 7 } and appending the HD-fragment D 1.1 below this node c.In other words, the HD-fragment D 1 is precisely HD D. Fast Parallel Hypertree Decompositions 1:27 Call 1.1 of function Decomp with parameters H .E = {R 7 , R 8 , R 9 , R 10 }, H .Sp = ∅, and H .Conn = {x 1 , x 7 }.The ParentLoop (line 9 ) and the ChildLoop (line 17) will eventually choose λ p = {R 1 , R 7 } and λ c = {R 1 , R 8 }, respectively.Then function Decomp is called recursively with parameters x .E = {R 9 , R 10 }, x .Sp = ∅, and x .Conn = {x 1 , x 9 } on line 28 for the only component "below" and with parameters H comp up .E = {R 7 }, H comp up .Sp = {s 2 } with s 2 = {x 1 , x 8 , x 9 }, and H comp up .Conn = {x 1 , x 7 } on line 33 for the component "above." Call 1.1.1 of function Decomp with parameters H .E = {R 9 , R 10 }, H .Sp = ∅, and H .Conn = {x 1 , x 9 }.This call immediately returns "true", since we have reached the base case on lines 5 and 6.The corresponding HD-fragment D 1.1.1consists of a single node with λ-label {R 9 , R 10 }.Call 1.1.2 of function Decomp with parameters H .E = {R 7 }, H .Sp = {s 2 } with s 2 = {x 1 , x 8 , x 9 }, and H .Conn = {x 1 , x 7 }.In the ParentLoop, eventually, λ p = {R 1 , R 6 } will be chosen on line 9.In this case, comp down is actually all of H , and it clearly satisfies the size constraint on line 11.In the ChildLoop, λ c = {R 1 , R 7 } will eventually be chosen on line 17.It gives rise to χ c = {x 1 , x 7 , x 8 } on line 18 with a single [χ c ]-component comps c [i] = {s 2 }.Function Decomp will therefore be called on line 28 with parameters H x .E = ∅, H x .Sp = {s 2 }, and H x .Conn = {x 1 , x 8 }.This call (referred to as Call 1.1.2.1) returns "true", since we now have the base case on lines 7 and 8.
and H comp up .Conn = {x 1 , x 7 }.This leads to the Call 1.1.2.2 of function Decomp, which returns "true", since we again have the base case on lines 7 and 8.In total, Call 1.1.2 of function Decomp is successful and the corresponding HD-fragment D 1.1.2consists of two nodes: the node with λ-label {R 1 , R 7 } and its child node with λ-label {s 2 } We can now also construct the HD-fragment D 1.1 of the successful Call 1.1 of function Decomp.More precisely, HD-fragment D 1.1 is obtained by taking HD-fragment D 1.1.2, replacing the leaf node with λ-label {s 2 } by the node c with λ c = {R 1 , R 8 } and χ c = {x 1 , x 8 , x 9 } from Call 1.1 and appending the HD-fragment D 1.1.1below this node c.The resulting HD-fragment D 1.1 is shown in Figure 5(b).That is, D 1.1 is the subtree consisting of the bottom 3 nodes u 6 , u 7 , and u 8 , of the final HD D displayed in Figure 5(a).

2 on line 25 .
[χ c ]components computed on line 35 and the parameters supplied to the recursive calls of function Decomp (on lines 39 and 45) remain the same as without the restriction on the search space.Hence, function Decomp will ultimately return the value true also if we choose λ p on line 23.It remains to show that λ p and λ p indeed give rise to the same component comp down .To avoid confusion, let us write comp down to denote a [λ p ]-component and comp down to denote a [λ p ]-component.Let comp down be the unique [λ p ]-component that satisfies the condition |comps p [i]| > |H .E |+ |H .Sp | We have λ p ⊆ λ p .Decreasing a set can only increase the corresponding components.Hence, there exists a [λ p ]-component, call it comp down with comp down ⊆ comp down .We have to show that comp down = comp down holds.The set comp down consists of the edges and special edges of the [χ c ]-components contained in comp down (denoted as comps c in the algorithm), and the edges and special edges covered by χ c .Let us refer to these [χ c ]-components as C 1 , . . .,C .By Corollary 3.12, comp down is both a [λ(p)]component and a [χ (p)]-component.Hence, by Lemma 3.11, these [χ (c)]-components are at the same time the [λ c ]components contained in comp down .And the edges and special edges covered by χ c are of course also covered by λ c .Likewise, comp down consists of the (special) edges of the [λ c ]components contained in comp down plus the (special) edges covered by λ c .

Fig. 6 .
Fig. 6.Comparison of solved instances (green) and unsolved instances (red), relative to their edge and vertex size.

Fig. 7 .
Fig.7.Study of log-k-decomp scaling behaviour w.r.t. the number of processing cores used.For computing the averages, we only consider instances (combination of graph and width) where the sequential case (1 core) took more than 100 ms to solve.This study involved 465 * 5 = 2, 325 instances, namely every combination of an instances from HB larдe and a width between 2 and 6.
At the heart of log-k-decomp is the recursive function Decomp: It takes as input a connection ALGORITHM 1: log-k-decomp Type: ConnSub=(E: Edge set, Sp: Special Edge set, Conn: Vertex set ) Input: H : Connected Hypergraph Parameter :k: width parameter Output: true if hw of H ≤ k, else false begin H comp ConnSub(E: H , Sp: ∅, Conn: ∅) p ]-components of H if ∃i s.t.|comps p [i]| > |H .E |+ |H .Sp | 2 then comp down comps p [i] found child comp.else continue ParentLoop if V (comp down ) ∩ H .Conn λ p then continue ParentLoop connect.check It is convenient to first prove the following claim: Claim A. In every call of function Decomp in Algorithm 1 with parameter H it is guaranteed thatH .Conn ⊆ V (H ) holds with V (H ) = H .E) ∪ H .Sp).Proof of Claim A. The proof is by induction on the call depth of the recursive function Decomp.induction begin.The top-level call of function Decomp on line 3 is using the connection subhypergraph H comp as the parameter, where H comp .Conn is defined as ∅.Hence, H comp .Conn ⊆ V (H comp ) trivially holds.induction step.Suppose that Claim A holds for every call of function Decomp down to some call level n and suppose that function Decomp is called recursively during execution of Decomp at call level n.Suppose that this execution of Decomp is with parameter H .The only places where function Decomp is called recursively are lines 28 and 33.More specifically, Decomp is called with parameter H x on line 28 and with parameter H comp up on line 33.We have to show that both H x .Conn ⊆ V (H x ) (on line 28) and H comp up .Conn ⊆ V (H comp up ) (on line 33) hold.On line 28, the condition is trivially fulfilled, since we have that H x .Conn = Conn x , and Conn x is defined on line 26 as Conn 2, the only non-trivial part being Condition 5: We have to verify H .Conn ⊆ χ (u).By Claim A above, we know that in every call of function Decomp, H .Conn is a subset of the vertices in H .E∪H .Sp.Now in case |H .E|+|H .Sp| = 1 holds, we have H .E ∪ H .Sp = { f } for a single (special) edge f and, therefore, χ (u) = f = H .E ∪ H .Sp .Hence, we indeed have H .Conn ⊆ χ (u).induction step.Now suppose that |H .E| + |H .Sp| > 1 and that function Decomp returns "true." This means that one of the return-statements in lines 6, 8, or 35 is executed.Actually, line 8 can be excluded for |H .E| + |H .Sp| > 1.Now consider the remaining two lines 6 and 35.If the returnstatement on line 6 is executed, then we have |H .E| ≤ k and |H .Sp| = 0.In this case, analogously to the induction begin, the desired HD consists of a single node u with λ(u) = H .E and χ (u) = H .E.Again, all conditions of an HD according to Definition 3.2 are easy to verify; in particular, the proof argument for Condition 5 is the same as above.
EdgeCount In EdgeCount we simply use the number of edges of the hypergraph |E(H )| as the measure of complexity.WeightedCount The WeightedCount metric is characterised by the formula |E(H )|

Table 1 .
Study of Two Hybrid Methods of log-k-decomp on HB larдe and a Comparison with NewDetKDecomp and HtdLEO for Reference Bolded entries represent the best value among the compared methods, where 'best' is understood as the highest value in the solved column and the lowest value in the run time column.

Table 2 .
Comparison of Prior Methods and log-k-decomp: Number of Cases Solved and Runtimes (Seconds) to Find Optimal-width HDs

Table 3 .
Extension of Table 2 with Runtimes for HtdLEO Extended to 10 Hours

Table 4 .
Comparison of the Decomposition Methods by How Many Instances Were Solved for a Specific WidthBolded entries represent the highest value achieved among the three compared systems for a given width, thus marking the best achieved result.

Table 5 .
Comparison of the Decomposition Methods by the Upper Bounds It Could ProvideProblem to solve Virtual Best log-k-decomp (Hybrid) NewDetKDecomp log-k-decomp Note that HtdLEO is not being explicitly considered here, since it directly computes the optimal width.Thus it would have the number 2,544-its number of solved instances-in each row.Bolded entries represent the highest value achieved among the three compared systems for a given problem, thus marking the best achieved result.