TOrPEDO: witnessing model correctness with topological proofs

Model design is not a linear, one-shot process. It proceeds through refinements and revisions. To effectively support developers in generating model refinements and revisions, it is desirable to have some automated support to verify evolvable models. To address this problem, we recently proposed to adopt topological proofs, which are slices of the original model that witness property satisfaction. We implemented TOrPEDO, a framework that provides automated support for using topological proofs during model design. Our results showed that topological proofs are significantly smaller than the original models, and that, in most of the cases, they allow the property to be re-verified by relying only on a simple syntactic check. However, our results also show that the procedure that computes topological proofs, which requires extracting unsatisfiable cores of LTL formulae, is computationally expensive. For this reason, TOrPEDO currently handles models with a small dimension. With the intent of providing practical and efficient support for flexible model design and wider adoption of our framework, in this paper, we propose an enhanced—re-engineered—version of TOrPEDO. The new version of TOrPEDO relies on a novel procedure to extract topological proofs, which has so far represented the bottleneck of TOrPEDO performances. We implemented our procedure within TOrPEDO by considering Partial Kripke Structures (PKSs) and Linear-time Temporal Logic (LTL): two widely used formalisms to express models with uncertain parts and their properties. To extract topological proofs, the new version of TOrPEDO converts the LTL formulae into an SMT instance and reuses an existing SMT solver (e.g., Microsoft Z3) to compute an unsatisfiable core. Then, the unsatisfiable core returned by the SMT solver is automatically processed to generate the topological proof. We evaluated TOrPEDO by assessing (i) how does the size of the proofs generated by TOrPEDO compares to the size of the models being analyzed; and (ii) how frequently the use of the topological proof returned by TOrPEDO avoids re-executing the model checker. Our results show that TOrPEDO provides proofs that are smaller (≈\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\approx $$\end{document} 60%) than their respective initial models effectively supporting designers in creating model revisions. In a significant number of cases (≈\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\approx $$\end{document} 79%), the topological proofs returned by TOrPEDO enable assessing the property satisfaction without re-running the model checker. We evaluated our new version of TOrPEDO by assessing (i) how it compares to the previous one; and (ii) how useful it is in supporting the evaluation of alternative design choices of (small) model instances in applied domains. The results show that the new version of TOrPEDO is significantly more efficient than the previous one and can compute topological proofs for models with less than 40 states within two hours. The topological proofs and counterexamples provided by TOrPEDO are useful to support the development of alternative design choices of (small) model instances in applied domains.


Introduction
One of the goals of formal methods is to provide automated verification tools that support designers in producing models that satisfy a set of properties of interest. Designers benefit from automated support in two cases: (i) when their models do not satisfy the properties of interest, and (ii) when they do satisfy such properties. While model checkers provide support in the first case-by producing counterexamples that explain why properties are not satisfied-theorem provers sustain the second case-by justifying why properties are satisfied. Theorem provers usually rely on some form of deductive mechanism that, given a set of axioms, iteratively applies a set of rules until a theorem is proved (e.g., [PPZ01,PZ01]). The proof consists of the sequence of deductive rules applied to prove the theorem. Even for simple models, proving the theorem requires the use of a considerable number of deductive rules, leading to complex proofs. This makes deductive proofs difficult to understand and hardly relatable to the designer's modeling choices. In addition, after the models are changed and model revisions are created, deductive proofs do not provide effective support for the automated verification of the model revisions.
To tackle this problem, we recently proposed the novel notion of topological proof (TP) [MRB20], which overcomes the complexity of deductive proofs and is designed to make proofs useful for the iterative verification of model revisions. A topological proof is a slice of the original model that witnesses which part of the model impacts the property satisfaction. Knowing which slice of the model impacts the property satisfaction can guide designers in refining and revising their models as it helps to select the parts of the model to be changed (or maintained). Furthermore, a topological proof can reduce the cost of formal verification. Indeed, in many cases, after a model changes, it is possible to assess the satisfaction of a property by only verifying whether the new version of the model preserves the topological proof, as opposed to re-executing the model checking procedure, which-typically-is computationally more expensive.
In our previous work [MRB20], we formally defined topological proofs by considering Partial Kripke Structures (PKSs) [BG99] and Linear-time Temporal Logic (LTL) to respectively express the model and the properties of interest. While it is possible to consider other modeling formalisms, we chose Partial Kripke Structures since (i) they are used in requirement elicitation to reason about system behavior from different points of view [EC01, BCE + 06], and are a common theoretical reference language used in the formal method community for the specification of uncertain models (e.g, [GJ03,BG99,GP09,BG00]); (ii) other modeling formalisms commonly used in software development [FUMK06,Uch09], such as Modal Transition Systems [LT88] (MTSs), can be converted into Partial Kripke Structures through a simple transformation [GJ03] making our solution easily applicable to those models; and (iii) Kripke Structures (KSs) are particular instances of Partial Kripke Structures that represent complete models. As such, our definitions can also be applied to models that do not contain uncertain parts. We chose Linear-time Temporal Logic since it is a standard logic used to express properties that should hold on Partial Kripke Structures.
To support the use of topological proofs during model design, we proposed TOrPEDO (TOpological Proof drivEn Development framewOrk) [MRB20], a novel automated verification framework, that: (i) supports Partial Kripke Structures and Linear-time Temporal Logic; (ii) allows performing analysis and verification in the context of models in which "incompleteness" represents a conceptual uncertainty; (iii) guides refinements and revisions through complementary outputs: counterexamples and proofs; and (iv) when the system is completely specified allows understanding which changes impact or not the satisfaction of certain properties.
There exists two variants of TOrPEDO: TOrPEDO-MUP and TOrPEDO-SMT. TOrPEDO-MUP was developed in our previous work [MRB20], TOrPEDO-SMT is part of the contribution of this work.
In our previous work, we implemented TOrPEDO using NuSMV [CCG + 02], i.e., an efficient and widely known model checker, and PLTL-MUP [SGT13], i.e., a tool that enables to compute a minimal subset of unsatisfiable LTL formulae from an unsatisfiable set of LTL formulae. We identify this version of TOrPEDO as TOrPEDO-MUP in the rest of the work. We evaluated TOrPEDO-MUP by considering a set of examples coming from literature including both completely specified and partially specified models. We evaluated TOrPEDO-MUP by assessing how the size of the proofs generated by TOrPEDO compares to the size of the models being analyzed (RQ1) and how frequently the use of the topological proofs returned by TOrPEDO-MUP avoids re-executing the model checker (RQ2). Our results show that topological proofs are ≈60% smaller than the original models and that in ≈79% of the cases topological proofs allow assessing the property satisfaction without re-executing the model checker. However, our results also show that the procedure that computes topological proofs is computationally expensive. Specifically, the procedure to extract unsatisfiable cores of LTL formulae, which is used to compute topological proofs, presents serious drawbacks. For this reason, TOrPEDO-MUP can currently handle only models of small dimensions. Table 1. Natural language and LTL formulation of the requirements of the vacuum-cleaner robot Textual Requirements LTL formulae φ 1 : the robot is drawing dust (suck) only if it has reached the cleaning site.
φ 4 ≡ ((¬suck) W(move ∧ (¬suck))) G and W are the "globally" and "weak until" LTL operators To provide a more practical and efficient support for flexible model design, in this paper we propose a new version of TOrPEDO, hereon called TOrPEDO-SMT. TOrPEDO-SMT reduces the computational cost required to compute topological proofs. It relies on a novel procedure to extract topological proofs. This procedure converts LTL formulae into a Satisfiability Modulo Theories (SMT) problem instance. We reuse existing techniques to convert LTL formulae into an SMT problem [SLJ + 06]. Specifically, since our goal is to reduce the computational cost of computing topological proofs, we implemented our translation by relying on a novel encoding [BKR15,PKRB20] based on Bit-Vectors. According to the authors, such encoding provides significantly better results when compared to other existing encodings. Then, TOrPEDO-SMT reuses an existing efficient SMT solver (e.g., Microsoft Z3 [DMB08]) to compute the unsatisfiable core of the SMT instance. Finally, the unsatisfiable core is automatically analyzed to extract the topological proof, which is returned to the model designer.
We evaluated TOrPEDO-SMT by assessing how efficient it is in analyzing models and how it compares to TOrPEDO-MUP (RQ3). TOrPEDO-SMT was able to compute topological proofs for models with less than 40 states within two hours. Furthermore, the results show that TOrPEDO-SMT is significantly more efficient than TOrPEDO-MUP. Finally, we assessed how useful TOrPEDO-SMT is in supporting the evaluation of alternative design choices of (small) model instances (RQ4). Our results show that the topological proofs and counterexamples provided by TOrPEDO effectively supported the development of a model of a small gene regulatory network.
Organization. Section 2 discusses the background. Section 3 describes TOrPEDO. Sections 4 and 5 present the theoretical results and the algorithms that support TOrPEDO. Section 6 evaluates the achieved results. Section 7 discusses related work. Section 8 presents our conclusions.

Running example and background notation
To illustrate TOrPEDO, we use the running example presented in Fig. 1 and Table 1; it contains a simple model describing the behavior of a vacuum-cleaner robot that has to satisfy the requirements described in Table 1. Section 2.1 introduces Partial Kripke Structures (PKS)-the modeling formalism considered in this work-and describes how the model design proceeds through refinements and revisions. Section 2.2 describes Linear-time Temporal Logic (LTL), the formalism used to express the properties of interest and its three-valued semantics. Section 2.3 shows how to model-check the satisfaction of LTL properties on PKSs.

Partial Kripke structures
The model of the vacuum-cleaner robot is represented using a Partial Kripke Structure (PKS) in Fig. 1a. PKSs are state machines that can be adopted when the values of given propositions on selected states is uncertain.
where: S is a set of states; R ⊆ S × S is a left-total transition relation on S ; S 0 is a set of initial states; AP is a set of atomic propositions; L : S × AP → { , ?, ⊥} is a function that, for each state in S , associates a truth value to every atomic proposition in AP

Fig. 1. Example models of a vacuum-cleaner robot
A PKS represents a system as a set of states and transitions between these states. The size |M | of a PKS M S , R, S 0 , AP , L is |AP | * |S | + |R| + |S 0 |. We defined the size of the PKS as the sum of (a) the number of atomic propositions assignments, that is obtained as the product between the cardinalities of the sets of the atomic propositions and the states of the PKS; (b) the cardinality of the set of the transitions of the PKS; and (c) the cardinality of the set of the initial states of the PKS. Our definition follows classical definitions used for KS (e.g., [KG96]) and ensures that the size of the PKS increases with the number of the states, the atomic propositions, and the transitions of the PKS The PKS of the vacuum-cleaner robot presented in Fig. 1a is defined over two atomic propositions representing actions that a robot can perform: move, i.e., the agent travels to the cleaning site; suck, i.e., the agent is drawing the dust, and two atomic propositions representing conditions that can trigger actions: on, true when the robot is turned on; reached, true when the robot has reached the cleaning site. The state O F F represents the robot being shut down, I DL E the robot being turned on w.r.t. a cleaning call, M OV I N G the robot reaching the cleaning site, and C L E AN I N G the robot performing its duty. Each state is labeled with the actions move and suck and the conditions on and reached. Let α and s be respectively an atomic proposition and a state. We use the notation: α to indicate that α is true when the robot is in state s; α ⊥ to indicate that α is false when the robot is in state s; α ? to indicate that there is uncertainty on whether α is true or false when the robot is in state s.
Model design proceeds though refinements and revisions. We will describe how model refinements and revisions are used in incremental model design in Sect. 3. Refinements assign either a or a ⊥ value to an atomic proposition α, in a state s, such that L(s, α) ?.
We use the notation M M rf to indicate that M rf is a refinement of M . For example, the PKS in Fig. 1b is a refinement of the PKS in Fig. 1a, obtained by changing the value of the proposition reached into ⊥ in state I DL E and the value of the proposition move into in state C L E AN I N G.  This definition of revision requires the model M rv to contain at least the same propositions of M , i.e., AP ⊆ AP rv . However, the definition does not force any relation between S , R, S 0 , L and S rv , R rv , S rv ,0 , L rv , respectively. Therefore, a revision M rv can be obtained from M by arbitrarily changing its initial states, states, and transitions. Intuitively, this means that the only constraint the designer has to respect during a revision is not to remove propositions from the set of atomic propositions. This condition is necessary to ensure that any property that can be evaluated on M can also be evaluated on M rv , i.e., every atomic proposition has a value in each of the states of the PKS. Instead, the deactivation of a proposition can be simulated by associating its value to ⊥ in all the states of M rv . A refinement is a particular type of revision; for example, the PKS in Fig. 1b is an example of a revision of the PKS in Fig. 1a.
Proof Sketch Since the PKSs M and M rf share the atomic proposition set AP , the condition AP ⊆ AP rv of Definition 2.4 is satisfied.

Linear-time temporal logic and three-valued semantics
The properties that the model of the vacuum-cleaner robot should satisfy are expressed in Linear-time Temporal Logic (LTL) in Table 1. LTL formulae combine atomic propositions with the Boolean connectors "and" (∧) and "not" (¬) and the temporal modalities "next" (X ) and "until" (U).
Definition 2.5 ([Pnu77]) Given a set AP of atomic propositions (with p ∈ AP ), an LTL formula φ is formed according to the following grammar: where φ 1 and φ 2 are LTL formulae.
The "or" (∨), "implication" (→) and "equivalence" (↔) Boolean operators are derived using the operators "and" (∧) and "not" (¬). Furthermore, the temporal operators "eventually" (F), "globally" (G) and "weak until" (W), are derived using the other temporal operators as usual. For example, φ 1 W φ 2 is defined using the "until" (U) and the "globally" (G) operators as (φ 1 U φ 2 ) ∨ G φ 1 . For KSs, we consider the classical LTL semantics [M | φ] over infinite words, which associates to a model M and a formula φ a truth value in the set {⊥, } (see for example [BK08]). For PKS, instead, the three-valued LTL semantics [BG99] [M | φ] associates to a model M and a formula φ a truth value in the set {⊥, ?, }, being based on the information ordering > ? > ⊥. The three-valued LTL semantics is defined by considering paths on the model M . A path π is an infinite sequence of states s 0 , s 1 , . . . such that, for all i ≥ 0, (s i , s i+1 ) ∈ R. We use the notation π i to indicate the infinite sub-sequence of π that starts at position i , and Path(s) to indicate the set of paths that start in the state s. . . be a path, and φ be an LTL formula. Then, the three-valued semantics [(M , π) | φ] is defined inductively as follows: Let M S , R, S 0 , AP , L be a PKS, and φ be an LTL formula.
The comp operator maps to ⊥, ⊥ to , and ? to ?. The minimum and the maximum functions are defined by considering the order > ? > ⊥. The minimum and the maximum functions are extended to sets by considering min(∅)= and max(∅)=⊥.
For example, consider the path O F F, I DL E, M OV I N G ω of the PKS in Fig. 1a, where M OV I N G ω indicates that the state M OV I N G is entered infinitely often. The three-valued semantics associates to the LTL property φ 1 the value ?, since its satisfaction depends on the value assigned to the proposition reached in the states I DL E and M OV I N G.

Model checking
Checking KSs with respect to LTL properties can be done by using classical model checking procedures. We assume that the function Check returns a tuple res, c , where res is the model checking result in { , ⊥} and c is the counterexample if res ⊥, else an empty sequence. Note that the model checking problem of a property φ on a KS M can be reduced to the satisfiability problem of the LTL formula M ∧ ¬φ, where M represents the behaviors of model M .
. Checking a PKS M with respect to an LTL property φ considering the three-valued semantics is done by performing twice the classical model checking procedure for KSs [BG00], one considering an optimistic approximation M opt and one considering a pessimistic approximation M pes . These two procedures consider the LTL formula φ neg F(φ), where F transforms φ with the following steps: (i) negation of φ; (ii) conversion of ¬φ in negation normal form; 1 (iii) replacement of every subformula ¬α, where α is an atomic proposition, with a new proposition α.
To create the optimistic and pessimistic approximations M opt and M pes , the PKS M Atomic propositions in AP are called complement-closed propositions. Function L c is such that, for all s ∈ S and α ∈ AP , L c (s, α) L(s, α) and, for all s ∈ S and α ∈ AP , L c (s, α) comp(L(s, α)). The complement-closed PKS of the vacuum-cleaner agent in Fig. 1a   • for all s ∈ S , α ∈ AP c , and L c (s, α) ∈ { , ⊥}, it holds that L pes (s, α) L c (s, α) (resp. L opt (s, α) L c (s, α)), and • for all s ∈ S , α ∈ AP c , and L c (s, α) ?, it holds that L pes (s, α) ⊥ (resp. L opt (s, α) ).
Let A be a KS and φ be an LTL formula, A | * φ is true if A does not contain any path that satisfies the formula F(φ).
We call Check * the function that computes the result of the operator | * . Three-valued model checking tools take as input the KS M and the LTL property φ, and return a tuple res, c , where res is the model checking result in { , ?, ⊥}, and c can be: • an empty sequence (when M satisfies φ), • a definitive-counterexample (when M violates φ), or • a possible-counterexample (when M possibly-satisfies φ).
Both counterexamples indicate behaviors that violate the properties of interest: definitive-counterexamples depend on already performed design choices, possible-counterexamples are based also on uncertain actions and conditions. Intuitively, in the construction of the optimistic completion M opt , the algorithm "tries its best" to build a KS which satisfies φ. If a violating behavior is found in M opt , then a definitive-counterexample is returned since the property φ does not hold. The presence of a violating behavior in M opt can be checked by verifying whether M opt ∧ φ neg is satisfiable, where M opt represents the behaviors of model M opt and φ neg F(φ). If M opt ∧ φ neg is satisfiable, then there exists a behavior that satisfies φ neg , that is it violates φ. Therefore, such behavior is a definitive counterexample. Viceversa, in the construction of the pessimistic completion M pes , the three-valued model checker "tries its best" to construct a KS that violates φ. If no violating behavior is found in M pes , then M | φ. The presence of a violating behavior in M pes can be checked by verifying whether M pes ∧φ neg is satisfiable, where M pes represents the behaviors of model M pes . If M pes ∧ φ neg is not satisfiable, then it does not exists any behavior that satisfies φ neg , that is φ is satisfied. Otherwise, it could be the case where there exists some completion in which φ holds and others in which it does not hold. In this case, the three-valued model checker returns ?.
For example, the PKS represented in Fig. 1a possibly satisfies LTL property φ 1 . The path O F F, I DL E, M OV I N G ω is a possible-counterexample for this property.
While definitive and possible counterexamples provide information when the property of interest is violated, or possible violated, they do not provide any information that explains why the model satisfies or possibly satisfies a property of interest. To tackle this problem, we propose TOrPEDO.

The topological proof driven development framework
The TOpological Proof drivEn Development framewOrk (TOrPEDO) is a development framework which supports the use of topological proofs (TPs) during the model design. The TOrPEDO framework is illustrated in Fig. 2 and carries out verification in four phases: initial design, analysis, revision, and re-check.
Initial design ( 1 ). The model M of the system is expressed using a PKS ( 1 ), which can be generated from other languages (e.g., MTS), along with the property of interest φ, expressed using LTL ( 2 ).
Running example. The model in Fig. 1a contains an example of PKS with an initial design for the vacuumcleaner robot. Table 1 contains the properties of interests for this model.
Analysis ( 2 ). TOrPEDO provides automated analysis support, which includes the following elements:   Table 2. Results provided by TOrPEDO for properties φ 1 , φ 2 , φ 3 and φ 4 . , ⊥ and ? indicate that the property is satisfied, violated and possibly satisfied.
φ 1 ? The automated analysis support relies on two components: the model checker and the topological proof extractor. The model checker verifies whether a property is satisfied or violated by the current model and is implemented by reusing existing algorithms from the literature, such as the one presented in Sect. 2. The topological proof extractor computes the topological proof and is discussed in Sect. 3.1.
In the following we will use the notation x -topological proofs or x -TP to respectively indicate definitivetopological or possible-topological proofs.
Running example. The results returned by TOrPEDO for the different properties in our motivating example are presented in Table 2. Property φ 2 is satisfied, φ 3 is not. In those cases, TOrPEDO returns respectively a definitive-proof and a definitive-counterexample. Since φ 1 and φ 4 are possibly satisfied, in both cases a possiblecounterexample and a possible-topological proof are returned. For φ 1 , the possible-counterexample shows a run that may violate the property of interest. The possible-topological proof for φ 1 in Table 2 shows that if O F F remains the only initial state (TPI), reached still holds in C L E AN I N G, and suck does not hold in O F F and I DL E, while unknown in M OV I N G (TPP), property φ 1 remains possibly satisfied. In addition, all transitions must be preserved (TPT). 2 Note that the proof highlights portions of the model that influence the property satisfaction. For example, by inspecting the proof, the designer understands that she can change the value of the proposition reached in all the states of the PKS, with the exception of the state C L E AN I N G, without making the property violated. Running example. The designer may want to propose a revision that still does not violate properties φ 1 , φ 2 , and φ 4 . Thus, she changes the values of some atomic propositions: move becomes in state C L E AN I N G and reached becomes ⊥ in state I DL E. Fig. 1b contains the revision of the PKS in Fig. 1a obtained by applying these changes. Since φ 1 , φ 2 , and φ 4 were previously not violated, TOrPEDO performs the re-check phase for each property.
Re-check ( 4 ). The automated verification tool provided by TOrPEDO checks whether all the changes in the current model revision are compliant with the x -TPs ( 10 ), i.e., changes applied to the revised model do not include parts that had to be preserved according to the x -topological proof. If a property of interest is (possibly) satisfied in a previous model, and the revision of the model is compliant with the property x -TP, the designer has the guarantee that the property is (possibly) satisfied in the revision. Thus, she can perform another model revision round ( 7 ) or approve the current design ( 11 ). Otherwise, TOrPEDO re-executes the analysis ( 12 ).
Running example. In the vacuum-cleaner case, the revision in Fig. 1b passes the re-check and the designer proceeds to a new revision phase.

Topological proof extractor
We present two implementations for the topological proof extractor component: our previous implementation [MRB20], which is based on PLTL-MUP [SGT13], and a novel SMT-based procedure, which is part of the contribution of this work. The versions of TOrPEDO that use these two topological proof extractor components are named TOrPEDO-MUP are TOrPEDO-SMT. In the rest of this work, we will use TOrPEDO-MUP and TOrPEDO-SMT when referring to the specific solver used to extract topological proofs, TOrPEDO when indicating the general method.
The topological proof extractor employed during the analysis phase takes as input a PKS M , its optimistic or pessimistic approximation A and an LTL formula φ neg F(φ) (see Sect. 2.3) and returns a slice S of the KS. Figure 3 conceptually describes the components of the topological proof extractor used in both TOrPEDO-MUP and TOrPEDO-SMT: rectangular boxes with sharp corners represent the inputs and outputs of the topological proof extractor; rectangular boxes with rounded corners represent software components. Their background color indicates whether the components are part of TOrPEDO-MUP (i.e., gray), TOrPEDO-SMT (i.e., red), or both (i.e., white). The labels on the arrows describe the type of inputs and outputs of each component.
The components implement the steps of Algorithm 1: TOrPEDO employs the Sys2LTL procedure to convert the KS A and the LTL formula φ neg into a set C of LTL clauses (Line 2). The set of LTL clauses C contains: i) clauses C A encoding the behaviors of the model A and ii) the formula φ neg . Since φ is satisfied, none of the behaviors of the model satisfy φ neg ; since the behaviors that violate the property cannot occur, some of the clauses in C A must conflict with φ neg . Our GetUC procedure detects such clauses and stores them within the set of conflicting clauses C uc (Line 3). Finally, the GetTP component maps the conflicting clauses within C uc into the corresponding slice S of the KS A (Line 4). TOrPEDO-MUP and TOrPEDO-SMT differ in the implementation of the GetUC procedure. TOrPEDO-MUP implements it by using PLTL-MUP [SGT13], which returns the conflicting clauses (C uc ). TOrPEDO-SMT, instead, translates the LTL clauses into a set of clauses P in propositional logic (PL), maintaining a map between each LTL clause and the propositional logic clauses generated from it. Then, it exploits existing SMT-based solvers to compute the conflicting propositional logic clauses (PC). Finally, it uses the previously built map to detect the LTL clauses that generated the conflicting propositional logic clauses.
The two procedures are described in detail in Sects. 5.3 and 5.4. We evaluate TOrPEDO-MUP and TOrPEDO-SMT in Sect. 6.

Topological proofs: a formal definition
In this section, we introduce the notion of topological proof. The pursued proof consists of a set of clauses that specify certain topological properties of M ; these represent the portion of the model that explains how it satisfies (or possibly-satisfies) the imposed claim. Different kinds of clauses are defined next. • a Topological Proof Initial-states clause (TPI-clause), i.e., an element S 0 .
These clauses indicate topological properties of a PKS M . Informally, TPP-clauses constrain how states are labeled (L), TPT-clauses constrain how states are connected (R), and TPI-clauses constrain the initial states of the model (S 0 ). For example, let us consider in Table 2 the proof obtained for property φ 1 : is a TPP-clause that constrains the atomic proposition reached to be labeled as true ( ) in the state C L E AN I N G; We now define the notion of -related PKS that is then used to formally define a topological proof. ii) for every TPP-clause s, α, v ∈ , the condition s ∈ S rel and v L rel (s, α) holds; iii) for every TPT-clause s, T ∈ , the condition s ∈ S rel , T ⊆ S rel , and T {s i ∈ S rel |(s, s i ) ∈ R rel } holds; iv) for every TPI-clause S 0 ∈ , the condition S 0 S rel ,0 holds. Based on these observations, a topological proof is then defined as follows.

Intuitively, a PKS -related to M is a PKS
The operator ≥ assumes that values , ?, ⊥ are ordered considering the classical information ordering > ? > ⊥ among the truth values [BG99]. Intuitively, an x -topological proof for M ensures that every -related PKS M rel to M satisfies the property φ "at least as much as M does". We call -TP a definitive-topological proof and ?-TP a possible-topological proof. Intuitively, a definitive-topological proof for M ensures that every -related PKS M rel to M satisfies the property φ. A possible-topological proof for M ensures that every -related PKS M rel to M satisfies or possibly satisfies the property φ.
The size of an x -topological proof is defined as | | c∈ |c| where: Note that, for the PKS in Fig. 1a, Table 2 shows two ?-TPs for properties φ 1 and φ 4 (respectively of size 14 and 10), and one -TP for property φ 2 (of size 14).
To introduce the core property of topological proofs, we first present the notion of x -revision, a revision of the model that follows the constraints imposed by the topological proof. Note that, differently from the notion of -relation, which is defined by considering an arbitrary set of TP-clauses, the notion of x -revision is defined by considering the topological proof, which is a set of TP-clauses that follows the conditions specified in Definition 4.3.
Definition 4.4 Let M and M rv be two PKSs, let φ be an LTL property, and let be an x -TP for φ in M . Then, The x -revision M rv of M is such that M rv is -related to M ; this means that it is obtained by changing the model M while preserving the statements that are specified in the Intuitively, a x -revision of M is a PKS M rv obtained from M by changing any topological aspect that does not impact on the set of TP-clauses . Specifically, models can be changed as follows: 1. any transition whose source state is not the source state of a transition included in the TPT-clauses can be added or removed from the PKS; 2. any value of a proposition that is not constrained by a TPP-clause can be changed; 3. states can be added and removed if they are not constrained by any TPT-, TPP-, or TPI-clause; 4. initial states cannot be changed if contains a TPI-clause.
For example, consider the topological proof for the property φ 1 presented in Table 2. The PKS in Fig. 1a is an ? -revision of the PKS in Fig. 1b since the values of the atomic propositions reached in I DL E and move in C L E AN I N G are not constrained by any TPP-clause.
In the following, we claim and prove that, if M is a PKS that satisfies the property φ, then any -revision of M also satisfies φ. In other words, if the model satisfies φ and it is revised in accordance with its topological proof, then -revisions of the model are generated and the revised model also satisfies φ. If M is a PKS that possibly satisfies the property φ, then any ? -revision possibly satisfies or satisfies φ. This means that, if the model possibly satisfies φ and is revised considering its topological proof, then ? -revisions of the model are generated and these also satisfy/possibly-satisfy φ. Consequently, when the model is revised according to its topological proof, there is no need to run the model checker to verify that φ is not satisfied on the revised model.

Automated support
This section describes the algorithms that support TOrPEDO. Section 5.1 describes how the model checker and the topological proof extractor are integrated into the analysis phase ( 2 ) of TOrPEDO. Section 5.2 describes in detail the steps of TOrPEDO used to compute topological proofs (Algorithm 1). Section 5.3 describes the implementation used by TOrPEDO-MUP for computing topological proofs. Section 5.4 describes the implementation of the SMTbased algorithm used by TOrPEDO-SMT for computing topological proofs. Section 5.5 presents the implementation of the re-check ( 4 ) phase of TOrPEDO.

Analysis
To analyze a PKS M S , R, S 0 , AP , L ( 1 ), TOrPEDO uses the three-valued model checking framework based on Theorem 2.1. The model checking result is provided as output by the analysis phase of TOrPEDO, whose behavior is described in Algorithm 2.
The algorithm returns a tuple x , y , where x is the verification result and y is a set containing the counterexample, the topological proof or both of them. The algorithm first checks whether the optimistic approximation M opt of the PKS M satisfies property φ ( 2 , Line 2). For computing M opt our implementation follows the steps described in Sect. 2.3: it first computes the complement-closed PKS by computing the set of complement-closed atomic propositions and the function L c , and then, it computes the optimistic approximation, by calculating the functions L opt . We used the NuSMV model checker for implementing the Check * procedure. If the optimistic approximation M opt of the PKS M violates the property φ, the property is violated by the PKS and the definitivecounterexample c d ( 3 , ⊥-CE) is returned (Line 3). Otherwise, the algorithm checks whether the pessimistic approximation M pes of the PKS M satisfies property φ (Line 5). As for the case of M opt , for computing M pes our implementation follows the steps described in Sect. 2.3. If the PKS M satisfies property φ, the property is satisfied and the value is returned along with the definitive-topological proof ( 4 , -TP) computed by the Ctp KS procedure applied on the pessimistic approximation M pes and the property φ neg (Line 7). Otherwise, the property is possibly satisfied and the value ? is returned along with the possible-counterexample c p ( 5 , ?-CE) and the possible-topological proof ( 6 , ?-TP) computed by the Ctp KS procedure applied to M opt and φ neg (Line 9).

p(s)
The KS is initially in one of its initial states.
If the KS is in state s in the current instant, in the next instant it is in one of the successors s i of s.
, the atomic proposition α is true.
If the KS is in state s such that L A (s, α) ⊥, the atomic proposition α is false.
The KS is in at most one state at any time.

Extracting the topological proofs
The procedure Ctp KS (Compute Topological Proofs) to compute x -TPs detailed in Algorithm 1 takes as input a PKS M , its optimistic/pessimistic approximation, i.e., here denoted generically as the KS A, and an LTL formula φ neg F(φ)-satisfied in A (see Sect. 2). The three steps of the algorithm are described in the following.

Sys2LTL: Translating the KS and the property into an LTL formula
The KS A S , R, S 0 , AP c , L A and the LTL formula φ neg are used to generate a set of clauses C C A ∪{φ neg }. The clauses in C A encode the KS. Specifically, The sets of clauses c i , C N , CL and CL ⊥ , C R EG are defined as specified in Table 3. The initial clause c i specifies that the KS is initially in one of its initial states. The next state clauses in C N specify that if in the current instant the KS is in state s, in the next instant it is in one of the successors states s i of s. The labeling clauses in CL specify that if the KS is in state s such that L A (s, α) , the atomic proposition α is true. The labeling clauses in CL ⊥ specify that if the KS is in state s such that L A (s, α) ⊥, the atomic proposition α is false. Finally, the regularity clauses in C R EG specify that the KS is in at most one state at any time. Note that the clauses in C A are defined on the set of atomic propositions AP S AP c ∪ {p(s)|s ∈ S }, i.e., AP S includes an additional atomic proposition p(s) for each state s, which is true when the KS is in state s. The LTL formula ψ to be analyzed to check for conflicting clauses is defined as follows.   , α)) TPP The problem of model checking the KS A and the LTL formula φ neg is solved by checking the satisfiability of ψ (see Sect. 2). The number of clauses of ψ is the number of clauses in C A , which is, in the worst case, 1 + |S | + |S | × |AP c | + |S | × |S |, plus one, that is the clause generated from the formula φ neg .

GetTP: extracting the topological clauses
As detailed in Sect. 3, the GetTP component takes the subset C uc of the conflicting clauses as input. As we will discuss in Sects. 5.3 and 5.4, C uc {C uc,A ∪ {φ neg }), where C uc,A C uc,K S ∪ C uc,R EG such that C uc,K S ⊆ C K S and C uc,R EG ⊆ C R EG . Specifically, the set C uc,A contains the clauses regarding the KS (C uc,K S and C uc,R EG ) that made the formula ψ unsatisfiable. Since we are interested in clauses related to the KS that caused unsatisfiability, we extract the topological proof , whose topological proof clauses are obtained from the clauses in C uc,K S as specified in Table 4. Since the set of atomic propositions of A is AP c AP ∪ AP , in the table we use α for propositions in AP and α for propositions in AP . The table contains for each LTL clause its corresponding topological proof clause.
The elements in C uc,R EG are not considered in the TP computation as, given an LTL clause G(¬p(s) ∨ ¬p(s i )), either state s or s i is constrained by other TP-clauses that will be preserved in the model revisions.
Lemma 5.1 Let A be a KS and let φ neg be an LTL property. Let also ψ be the LTL formula computed in the step Sys2LTL of the algorithm, where C C A ∪ {φ neg } and C A C R EG ∪ C K S , and let ψ uc be an unsatisfiable core, where C uc C uc,A ∪ {φ neg } and C uc,A C uc,R EG ∪ C uc,K S . Then, if G(¬p(s) ∨ ¬p(s i )) ∈ C uc,R EG , either: (i) there exists an LTL clause in C uc,K S that constrains state s (or state s i ); or (ii) ψ uc , such that C uc C uc,A ∪ {φ neg } and C uc,A C uc,A \ {G(¬p(s) ∨ ¬p(s i ))}, is an UC of ψ uc .

Proof Sketch
We indicate G(¬p(s) ∨ ¬p(s i )) as τ (s, s i ). Assume per absurdum that conditions (i) and (ii) are violated, i.e., no LTL clause in C uc,K S constrains state s or s i (for condition (i)), and ψ uc is not an unsatisfiable core of ψ uc (for condition (ii)). Since ψ uc is not an unsatisfiable core of ψ uc , the LTL formula ψ uc where C uc C uc,A ∪ {φ neg } is satisfiable. Since C uc is satisfiable, C uc must also be satisfiable since C uc,A C uc,A ∪ {τ (s, s i )}. Indeed, since condition (i) is violated, it does not exist any LTL clause that constrains state s (or state s i ) and, in order to generate a contradiction, the added LTL clause must generate it using the LTL clauses obtained from the LTL property φ neg . This is a contradiction and proves our lemma.
The Analyze procedure in Algorithm 2 obtains a TP ( 4 , 6 ) for a PKS by first computing the related optimistic or pessimistic approximation (i.e., a KS) and then exploiting the computation of the TP for such KS. In the following we prove that our procedure is correct, i.e., given a PKS M and a property φ such that x it returns an x -topological proof for φ in M , that is every -related PKS M rel is such that We show that C uc,A ⊆ C B , i.e., the UC is also an UC for the LTL formula associated with the approximation B of the PKS N .
As C uc,A C uc,K S ∪ C uc,R EG this is equivalent to check (C uc,K S ∪ C uc,R EG ) ⊆ C B . By Lemma 5.2 we can avoid considering C uc,R EG . By construction (see Line 2 of Algorithm 1) any clause c ∈ C uc,K S belongs to one rule among C N , CL pes, , CL pes,⊥ or c c i : • if c c i then, by the rules in Table 4, there is a TPI-clause {S 0 } ∈ . By Definition 4.2, S 0 S 0,N . Thus, • if c ∈ C N then, by the rules in Table 4, there is a TPT-clause s, T ∈ where s ∈ S and T ⊆ R. By Definition 4.2, • if c ∈ CL A, or c ∈ CL A,⊥ , by rules in Table 4, there is a TPP-clause s, α, L(s, α) ∈ where s ∈ S and α ∈ AP . By Definition 4.2, L N (s, α) L(s, α). Thus, c ∈ C B since N is -related to M .
Since N is -related to M , it has preserved the elements of . Thus CA is also an UC of CB . It follows that [N | φ] . The proof from the case in which Analyze procedure returns the value ? and a ?-TP can be derived from the first case.

Computing the conflicting clauses in ψ with PLTL-MUP
We now describe the procedure used by TOrPEDO-MUP to extract the conflicting clauses of ψ. Recall that the set of clauses of the LTL formula ψ encodes the KS A and the property φ neg . As mentioned in Sect. 3.1, none of the behaviors of A satisfies the property φ neg since φ neg encodes the behaviors that violate φ which is satisfied by A. Therefore, ψ is unsatisfiable. The unsatisfiable core (UC) contains the set of conflicting clauses that lead to the contradiction. TOrPEDO-MUP uses PLTL-MUP [SGT13] to extract the unsatisfiable core of ψ. PLTL-MUP exploits Binary Decision Diagrams (BDDs) and a BDD-based theorem prover for LTL to compute the UC.
The function ψ uc GetUC(ψ) returns an unsatisfiable core ψ uc c∈C uc c of ψ. Specifically, it returns a subset of clauses C uc C uc,A ∪ {ψ}, where C uc,A C uc,K S ∪ C uc,R EG such that C uc,K S ⊆ C K S and C uc,R EG ⊆ C R EG .

Lemma 5.2 Let
A be a KS and let φ neg be an LTL property. Let C {C A ∪ {φ neg }) be the set of clauses generated from the KS A and the LTL formula φ neg , and ψ be the LTL formula computed in the step Sys2LTL of the algorithm. Then, any unsatisfiable core ψ uc of ψ is made by a subset of clauses C uc {C uc,A ∪ {φ neg }) such that C uc,A ⊆ C A .
Proof Sketch As the property φ neg is satisfied by M , the LTL formula ψ is unsatisfiable (see Sect. 2). Since C A encodes a KS, c∈C A c is satisfiable. If φ neg is satisfiable (i.e., it is not vacuously true or false), then the unsatisfiability is caused by the contradiction of some of the clauses in C A and the property φ neg , and as a consequence φ neg must be a part of the UC, C uc,A is not empty, and C uc,A ⊆ C A .
After detecting the set C uc of conflicting LTL clauses, the procedure described in Sect. 5.2.2 is used to generate the TP.

SMT-based extraction of topological proofs
This section describes the procedure to translate LTL clauses into propositional logic (PL) clauses (Sect. 5.4.1). Then, it presents the procedure used to detect conflicts among the propositional logic clauses (Sect. 5.4.2), and describes how the conflicting LTL clauses are extracted from the conflicting PL logic clauses (Sect. 5.4.3). Finally, it discusses the correctness of our procedure (Sect. 5.4.4).

LTL2PL: from linear time temporal logic to propositional logic
Recall that the LTL formula ψ-to be analyzed to check for conflicting clauses in C -is defined as follows. , also used in more recent works (e.g., [BKR15]). The technique is based on two observations: 1. an LTL formula ψ is satisfiable if there exists an infinite path that satisfies ψ; and 2. the infinite paths that satisfy LTL formulae can be represented as π s 0 , s 1 , . . . , s r −1 , (s r , s r +1 , . . . , s k ) ω , where (s r , s r +1 , . . . , s k ) ω indicates that, after state s k is left, the path restarts from s r . The infinite path is made by two parts: the prefix, i.e., s 0 , s 1 , . . . , s r −1 , and the ultimately periodic part s r , s r +1 , . . . , s k .
Building on these two observations, the idea is to check the satisfiability of ψ by: i) generating a propositional formula ϕ encoding all the possible ultimately periodic paths (up to length k ) that satisfy ψ and ii) checking for the satisfiability of ϕ. If ϕ is satisfiable, there exists an ultimately periodic path that satisfies ψ. Therefore, the LTL formula ψ is satisfiable. If ϕ is unsatisfiable and k is "big enough", then ψ is unsatisfiable. In Sect. 5.4.4 we will discuss the implications of the k selection on the correctness of our procedure.
The propositional formula ϕ is made by two parts, one encoding the infinite (ultimately periodic) paths, and one encoding the semantics of the formula ψ. These two parts are described in the following.
Encoding the Infinite (ultimately periodic) Paths. The encoding introduces the following variables and PL formulae, in which the symbols i , with i ∈ 0, 1, . . . are used to identify each PL formula.
• A set of loop selector variables (l 0 , l 1 ,. . . ,l k ). These are new fresh Boolean variables such that l i is true if the ultimately periodic part starts at position i . To enforce the fact that the ultimately periodic part starts from one state, only one of the loop selector variables must be true. This is enforced by the following formula: For each index i , the formula specifies that if l i is true, all the other loop selector variables are false. Furthermore, we force one of the loop selector variables to be true by adding the following formula: • A set of in loop variables (InLoop 0 , InLoop 1 ,. . . ,InLoop k ). These are new fresh Boolean variables such that InLoop i is true if the state s i is in the ultimately periodic part. To enforce the correspondence between the loop selector variables and the in loop variables, we add the following propositional formula.
This formula imposes that InLoop i is in the loop only if the ultimately periodic part starts at position i or it starts in a previous position, i.e., InLoop i−1 is true.
• The variable LoopExists. It is true if and only if a loop has been found. To force this relation we add the following formula.
Encoding the Semantics of the Formula ψ. To force the ultimately periodic path to satisfy the LTL formula, we need to encode how the formula is satisfied on the different states of the path. This should be specified according to the semantics of the LTL formula. In the following, we describe how each LTL clause c ∈ C is translated into a PL formula ϕ c . Let ψ 0 , ψ 1 , . . . , ψ h be the subformulae of a clause c of the LTL formula ψ. To represent the LTL clause c in PL, we introduce a Boolean variable ψ j i for each index i in 0, . . . , k + 1 and subformula ψ j of c. Each variable ψ j i represents the value of a subformula ψ j in the position i of the path. We consider indexes from 0, . . . , k + 1 since, to represent the ultimately periodic part of the path, we need to enforce that LTL subformulae satisfied by the state s k +1 correspond to the LTL subformulae satisfied by the state s r , i.e., in the first state of the ultimately periodic part of the path. We use the symbols ξ i , with i ∈ 0, 1, . . . to identify each one of the PL formulae. To enforce that the subformulae that hold in s k +1 are the same that hold in state s r , we add the following formula: Then, we enforce the LTL semantics by using the standard fix-point encoding specified in Table 5. For every subformula ψ 0 , ψ 1 , . . . , ψ h of c, a PL formula (i.e., ξ 1 , ξ 2 , . . . , ξ h+1 ) is created depending on the type of the constructs used in the subformula (see the PL labelled column of Table 5). The encoding of p, ¬ψ , X ψ, and ψ 1 ∧ ψ 2 directly follows from the LTL semantics. For ψ 1 U ψ 2 , the encoding specifies that ψ 1 U ψ 2 i ⇔ ( ψ 2 i ∨ ( ψ 1 i ∧ ψ 1 U ψ 2 i )) is true for every index i according to the semantics of the U temporal operator. However, this formula is vacously satisfied when ψ 1 i ∧ ψ 1 U ψ 2 i is true for every index i that belongs to the ultimately periodic part, but ψ 2 is never satisfied. To avoid this case, we need to ensure that ψ 2 eventually occurs. This is achieved by 1. the formula LoopExists ⇒ ( ψ 1 U ψ 2 k ⇒ F ψ 2 k ), specifying that whenever an ultimately periodic path is present (LoopExists is true), if ψ 1 U ψ 2 is satisfied in position k , F ψ 2 is also satisfied in position k ; 2. the formula i∈{0,...,k } ( F ψ 2 i ⇔ ( F ψ 2 i−1 ∨ (InLoop i ∧ ψ 2 i )), specifying that F ψ 2 holds in a position i of the loop (and therefore in position k ) if ψ 2 holds in a previous position of the loop; and 3. the formula F ψ 2 0 ⊥ forces F ψ 2 to be initially false.
The final PL formula ϕ c -that represents the LTL clause c-is obtained by combining the formula with the conjunction Boolean operator, i.e., ϕ c is defined as follows: Then, the satisfiability of ψ can be verified by checking the satisfiability of: Note that, since our goal is to reduce the computational cost required to compute topological proofs, we implemented the LTL2PL translation by relying on an encoding based on Bit-Vectors [BKR15]. According to some recent result, this encoding provides significant benefits with respect to existing tools [BKR15,PKRB20].

GetUC: computing the unsatisfiable core of a PL formula
To check whether the formula ϕ is satisfiable and to extract its unsatisfiable core, we employ the Z3 Theorem Prover [DMB08]; we selected this solver as it extracts unsatisfiable cores and it is an industry-strength tool, also awarded by ETAPS (Test of Time Award) [sig20] and ACM SIGPLAN (Programming Languages Software Award) [eta20].
Given a formula and a set of clauses, a.k.a. assumptions, Z3 checks whether the formula is satisfiable and identifies the clauses that are in contradiction. Specifically, given the PL formula ϕ and the set of PL clauses ϕ 1 , ϕ 2 , . . . , ϕ n , each one generated from an LTL clause c i ∈ C , Z 3.unsat core checks whether ϕ is satisfiable, and-in case it is not-it returns a subset of the set of the PL clauses {ϕ 1 , ϕ 2 , . . . , ϕ n } that lead to the contradiction.

GetTP: mapping the conflicting propositional clauses to LTL
The function GetTP aims at identifying-from the set of the conflicting PL clauses -the corresponding LTL clauses. Recall that, as discussed in Sect. 5.4.1, each clause within the set of PL clauses {ϕ 1 , ϕ 2 , . . . , ϕ n } processed by Z3 is obtained from an LTL clause c ∈ C . Therefore, given the set of the conflicting PL clauses , our algorithm computes the set C uc of conflicting LTL clauses by adding to C uc -for each PL formulae ϕ c in -the LTL clause c from which it was generated. After detecting the set C uc of conflicting LTL clauses, the procedure described in Sect. 5.2.2 is used to generate the TP.

Correctness
The unsatisfiable core contains the set of PL clauses of {ϕ 1 , ϕ 2 , . . . , ϕ n } that lead to a contradiction. As specified in Sect. 5.4.1, each PL clause in {ϕ 1 , ϕ 2 , . . . , ϕ n } corresponds to an LTL clause {c 1 , c 2 , . . . , c n } in C . The algorithm extracts from the subset C uc of C containing the LTL clauses of C corresponding to the PL clauses leading to the contradiction. The algorithm is correct if the LTL clauses in C uc are contradicting.
The bounded encoding presented in Sect. 5.4.1 finds ultimately periodic paths (up to length k , a.k.a. bound) that satisfy the formula ψ, but cannot prove their absence. To prove that such paths do not exist, it must be shown that a path that satisfies the LTL formula ψ cannot be longer than a certain bound, a.k.a. completeness threshold. Recall that, as for classical bounded model checking (BMC), the LTL formula ψ encodes the KS A and the LTL formula φ neg to be checked. Therefore, the completeness threshold can be identified by reusing existing approaches from the literature (see for example [SLJ + 06, CKOS04, KOS + 11]). Therefore, if the value of k is higher than the completeness threshold, there is formal guarantee that the LTL clauses in C uc are contradicting. Vice versa, if k is lower than the completeness threshold, the clauses in C uc may not be contradicting.
In practice, designers can initially choose a value for k that is reasonably large for the considered PKS and property. Then, they can increase or decrease the value of k depending on (a) the efficiency of the analysis component, and (b) how relevant is the soundness of the computed topological proofs for their application domain. • AP ⊆ AP rv ; • for every TPP-clause s, α, v ∈ , s ∈ S rv , v L rv (s, α); • for every TPT-clause s, T ∈ , s ∈ S rv , T ⊆ S rv , T {s i ∈ S rv |(s, s i ) ∈ R rv }; • for every TPI-clause S 0 ∈ , S 0 S rv ,0 .

Re-check
These conditions can be verified by a simple syntactic check on the PKS. For example, considering the x -TP for the property φ 2 presented in Table 2 of our vacuum-cleaner example, the re-check algorithm returns a true value for the revision M rv in Fig. 1b of the model M in Fig. 1a. The analysis and re-check algorithms assume that the three-valued LTL semantics is considered. An alternative semantics, called thorough LTL semantics [BG00], has been introduced to provide an evaluation of formulae that better reflects the natural intuition. It has been demonstrated that the two semantics coincide in the case of self-minimizing LTL formulae [GH05]. In this case, our results are correct also w.r.t. the thorough semantics. Note that, as shown in [GH05], most practically useful LTL formulae are self-minimizing.

Evaluation
We implemented TOrPEDO as a Scala stand alone application and made it available online [Tor20]. We implemented TOrPEDO-MUP and TOrPEDO-SMT by using NuSMV (v2.6.0) and Z3 (v4.4 build 2 rev 1). We evaluated how the analysis helps in creating models revisions, how frequently running the re-check algorithm allows the user to avoid the re-execution of the analysis algorithm from scratch. We also evaluated the efficiency of TOrPEDO and how it supports the development of (small) models. Specifically, we considered the following research questions:

RQ1:
How does the size of the proofs generated by the analysis algorithm of TOrPEDO-MUP compares to the size of the models being analyzed? (Sect. 6.1) RQ2: How frequently running the re-check algorithm of TOrPEDO-MUP allows to avoid the re-execution of the analysis algorithm from scratch? (Sect. 6.2) RQ3: How efficient is TOrPEDO in analyzing models and how does TOrPEDO-SMT compare to TOrPEDO-MUP? (Sect. 6.3) RQ4: How useful is TOrPEDO-SMT in supporting the designers in the model design on an example in the genomic domain? (Sect. 6.4) We used TOrPEDO-MUP for RQ1 and RQ2 since the analysis of the models considered for answering these questions did not require the most efficient version of our tool. We used TOrPEDO-SMT for RQ3 and RQ4. We run our experiments on a machine with processor Intel Core i5 3.2GHz and 32GB of memory.

Analysis support-RQ1
To answer RQ1, we checked how the size of the proofs generated by the analysis algorithm compares to the size of the models being analyzed. The topological proofs represent constraints that, if satisfied, ensure that the property is not violated (or possibly violated). As discussed in Sect. 4, to ensure that the property is not violated, the designers should not modify the parts of the model constrained by the proofs while creating model revisions. Therefore, the smaller the topological proofs are, the more useful they are, since more elements can be changed during the model revisions. In addition, smaller proofs allow for easier inspection, and this makes it easier to create revisions.  Table 7. Cardinalities |S |, |R|, |AP |, |?|, and |M | are those of the evaluated model M . | p | x is the size of proof p for a property p; x indicates if p is a -TP or a ?-TP. The column | p | x also reports percentage of the ratio between the size of the proof and the size of the model.   The goal of this research question is to assess how useful the computed topological proofs are. The usefulness is evaluated by comparing the size of the topological proofs to the size of the models being analyzed.
Dataset. We considered a set of 3 examples (callee, caller and caller-callee) proposed in the literature and used to evaluate χChek [ECD + 03]. For each of these examples, we consider the PKS representing the initial model, indicated using the name of the model followed by the index 1, and three model revisions, each indicated using the name of the model following an incremental index. For example, callee-1 indicates the initial PKS model for the callee example, while callee-2, callee-3, and callee-4 are three consecutive revisions of callee-1. Therefore, in our evaluation, we considered 12 PKS in total. Table 7 reports  We considered five properties for each example (see Table 6). These properties were inspired by the original properties and based on the LTL property patterns [DAC99]. 3 These properties are sufficiently diverse to support our experiments, since they use all the LTL operators. 4 Therefore, they enable us to assess the topological proofs computed starting from different LTL operators.
Methodology. We run the analysis component of TOrPEDO. We considered each of the 60 model-property combinations of our dataset. For each model-property combination, we executed the analysis component of TOrPEDO and we recorded its output (see Fig. 2).
For the cases in which TOrPEDO produced a topological proof, we also computed the size of the proof, as defined in Sect. 4. We compared the size of the PKS model and the size of the proofs returned by TOrPEDO.
Results. Table 7 summarizes the obtained results. We show the total size |M | of the model, and the size | p | of the proofs. The column | p | x also reports within rounded brackets the percentage of the ratio between the size of the proof and the size of the model. Cells labeled with the symbol × indicate that a property was not satisfied in that model and thus a proof was not produced by the analysis algorithm. The numbers indicating the size of the proofs are tagged with a subscript that indicates whether the property is satisfied (x ) or possibly satisfied (x ?). Proofs are ≈ 60% smaller than their respective initial models. Thus, we conclude that the proofs are significantly more concise than the original model, thus, enabling a flexible design.
The answer to RQ1 is that, on the considered models, TOrPEDO provides proofs that are ≈ 60% smaller than their respective initial models.

Re-check support-RQ2
To answer RQ2, we checked how the results output by the re-check algorithm were useful in producing PKSs revisions.
Dataset. We considered the same dataset used for RQ1. We assumed that, for each example, the designer produced revisions following the order specified in Table 8 (column Model), that is consecutive revisions are identified using consecutive indexes labeling the name of the same example. For example, the model callee-3 is a revision of the model callee-2 which, in turn, is a revision of the model callee-1. Since the design of callee-2 precedes the design of callee-3, we say that callee-2 is the previous model of callee-3. The other columns contain the different properties that have been analyzed for each category.
Methodology. We run the re-check component of TOrPEDO on the models and the properties of our dataset. We recorded the output of the re-check component. We assessed in how many cases the re-check component allowed designers to avoid re-runnning the analysis.
Results. Table 8 reports our results. A cell contains ✓ if the re-check was passed by the considered revised model, i.e., a true value was returned by the re-check algorithm, ✗ otherwise. For example, the ✓ symbol associated by the model callee-2 and the property φ 1 indicates that the re-check component confirmed that the revision callee-2 of callee-1 is an -revision of callee-1 by considering the topological proof for φ 1 in callee-1. The dash symbol -is used when the model of the corresponding line is not a revision (i.e., the first model of each category) or when the observed property was false in the previous model, i.e., an x -TP for the property was not produced. For example, the model caller-1 is not a revision, since it is the first model proposed for the caller example. Differently, the model caller-2 and the property ψ 2 are associated with the symbol -since, as specified in Table 7, the analysis component did not produce any topological proof of the property ψ 2 in the model caller-1. Therefore, the re-check could not be executed. We inspected the results produced by the re-check algorithm to evaluate their benefit in verifying if revisions were violating the proofs. Table 8 shows that, in ≈ 21% (number of cells labeled with the ✗ as a percentage of the number of cells labeled with symbols ✗ or ✓) of the cases, the TOrPEDO re-check notified the designer that the proposed revision violated some of the clauses contained in the -proof, while in ≈ 79% (number of cells labeled with the ✓ as a percentage of the number of cells labeled with symbols ✗ or ✓) the re-check allowed designers to avoid re-runnning the analysis (and thus the model checker).
The answer to RQ2 is that, on the considered models, in ≈ 79% the re-check component allowed designers to avoid re-runnning the analysis (and thus the model checker).

Efficiency-RQ3
To evaluate the efficiency of TOrPEDO we could not use the models of RQ1 and RQ2. Indeed, executing the analysis and re-check phases of TOrPEDO requires less than a minute for each property of each model. Therefore, analyzing the performances on these models does not provide significant practical results. For this reason, to evaluate the efficiency of TOrPEDO we analyzed a set of randomly generated models with increasing size. The analysis phase of TOrPEDO combines three-valued model checking and UCs computation. Three-valued model checking is as expensive as classical model checking [BG99], i.e., it is linear in the size of the model and exponential in the size of the property. UCs computation is FPSPACE complete [SHH12]. The re-check phase performs a simple syntactic check; the complexity of the re-check algorithm is linear in the size of the model. Therefore, we only analyze the efficiency of the analysis phase of TOrPEDO since the re-check requires negligible time compared to the analysis.
Dataset. We generated a set of random models with an increasing number of states (i.e., 10, 20, 30, and 40). The random models are generated from the grade crossing semaphore (GC) example ([BMS + 17]) starting from the GC model, and by iteratively duplicating the GC model and connecting the duplicated model with the initial model with randomly generated transitions. We considered two properties φ 1 and φ 2 that are respectively satisfied and possibly satisfied on the GC model and on the randomly generated models. Property φ 1 specifies that red lights up infinitely often ( red). Property φ 2 states that green lights up infinitely often ( green). Methodology. We run the analysis component of TOrPEDO-MUP and TOrPEDO-SMT tool by considering all the eight combinations made by a randomly generated model and a property. Note that, in our experiments we considered an extended version of PLTL-MUP, namely Hybrid, that improves the PLTL-MUP performances by combining it with TRP++UC [SGT13]. We set two hours as timeout for each run. For TOrPEDO-SMT, for each model and property, we set 86 as a value for the completeness bound k . We selected this value since it ensures the correctness of the result, i.e., we set its value by considering to the size of the recurrence diameter (the longest initialised loop-free path in the state graph) and the size of the Büchi automaton representing the negation of the property [CKOS05]. We recorded the output of the analysis component and the time required to produce the output result. Note that we did not report the time required by the model checker and the proof extraction separately since the cost of executing the model checker is negligible compared to the cost of computing the topological proof.
Results. Our results are reported in Fig. 4. The x-axis reports the number of states of each random model. The y-axis reports the time required by TOrPEDO-MUP and TOrPEDO-SMT for each run that finished within the timeout. TOrPEDO-MUP was not able to finish within the timeout for models above 20 states. For property φ 1 , TOrPEDO-MUP was able to process models with 10 and 20 states. For these models, the differences between the time required by TOrPEDO-MUP and the time required by TOrPEDO-SMT are approximately 3min and 36min, respectively. For property φ 2 , TOrPEDO-MUP was able to process only models with 10 states. For this model, the difference between the time required by TOrPEDO-MUP and the time required by TOrPEDO-SMT is approximately 2min. TOrPEDO-SMT was able to finish within the timeout for any model containing up to 40 states. The propositional logic formulae generated by TOrPEDO-SMT had approximately 1000 propositional logic operators. For the cases in which both TOrPEDO-MUP and TOrPEDO-SMT finished within the timeout, they required on average 15m and 1.4m, respectively.
The answer to RQ3 is that, on the considered models, TOrPEDO-SMT can verify within the timeout models which are double in size w.r.t. those which could be verified by TOrPEDO-MUP within the timeout. For the cases in which both the versions of our tool finished within the timeout, TOrPEDO-MUP and TOrPEDO-SMT required on average 15m and 1.4m, Therefore, TOrPEDO-SMT is significantly more efficient than TOrPEDO-MUP.

Usefulness-RQ4
To assess how useful TOrPEDO is in supporting the evaluation of design choices, we considered a (small) model example from the genomic domain, related to Gene Regulatory Networks (GRNs). GRNs are collections of molecular regulators, interacting with each other and governing the gene expression levels of mRNA and proteins. Typically, GRNs are represented using nodes (genes) connected by edges (inhibition/activation actions); edges may be weighted or unweighted using some coefficient (e.g., inferred using the Banjo method [YSW + 04]). GRNs are deduced from gene expression [Alu05] or ChIP-seq experiments data [SPM19] and are used to understand fundamental biological processes in the cell, as well as the pathogenesis of some diseases [ESDHK14,LXZW12], using genomic data integrated from several sources [BCMC21].
Verifying whether GRNs meet certain properties is a widely recognized problem [JCL + 09]. The correctness of the inferred networks may be performed manually (by comparison with public database, e.g., KEGG [KG00] and GO [ABB + 00]) or automatically (e.g., by using symbolic model checking techniques [GKD + 15, MDKG15]). We evaluated how TOrPEDO supports designers in establishing appropriate models of GRNs.
The MAPK Example. We considered a small network of the MAPK pathway 5 , inspired by [GKD + 15] and translated it into a PKS. Each of the six encompassed genes is represented by one proposition (i.e., Msg5, Fus3, Ste7, Far1, Ste11, and Dig1/2). The proposition is true if the gene is activated, false otherwise. Each configuration of the network describes the status of all the genes (activated/deactivated) and is represented as a state in the PKS. In total, we have 64 (2 6 ) states that represent all possible statuses of the genes. Transitions among PKS states encode how the status of the genes can change according to the behavior specified by the regulatory network. As initially the network can be in any configuration, all states of the PKS are initial states.
We considered two LTL properties that MAPK pathway should satisfy (i.e., φ 1 and φ 2 ) and one (φ 3 ) that it should not satisfy. These properties are inspired by the CTL specifications provided in [GKD + 15] and abstracted from KEGG's pathways characteristics [KG00], and then discussed with domain experts. Property φ 1 Fus3 → X (¬Dig1/2) is expressing that if Fus3 is activated, Dig1/2 will be inhibited immediately in the next step (i.e., Fus3 is a direct inhibitor of Dig1/2). Property φ 2 G(Msg5 ∨ Fus3) → F(¬Ste11) means that if globally either Msg5 or Fus3 is activated, finally Ste11 will be inhibited. Finally, property φ 3 G(Ste7 → F(Fus3)) is checking whether or not Ste7's activation will finally inhibit or promote the transcription of Fus3 cell cycle regulatory gene.
Methodology. As the initial model, we considered a version of the PKS having multiple uncertainty points, specifically: on Dig1/2 in two different states, on Ste11 in one state, and on Msg5 in another state. For the three observed properties, we simulated an incremental model design by running TOrPEDO on the initial model and we assess how useful are the artifacts produced by TOrPEDO to guide the model design. We articulate the timeline of experiments in Table 9.
Step 1, First, we run the analysis phase on the model against all three requirements, obtaining results for φ 1 , ? for φ 2 and ⊥ for φ 3 . • Step 2. We then inspected the topological proofs generated for φ 1 and φ 2 and, after consulting with domain experts, we proposed a first revision of the model that did not conflict with the listed clauses: namely, we assigned Msg5 ⊥ in states not shown in the proof. After this revision, it was sufficient to run the re-check procedure, to confirm the previous verification results. • Step 3. Similarly, by inspecting again the topological proofs, we proposed a second change in the model, producing a second revision that included new transitions among given states. In particular, adding new transitions allows us to envision a wider set of changes of configurations between the states of the GRN. In other words, we allowed additional gene activations and inhibitions with respect to the initially considered set. As this change does not involve any part of the model that is mentioned in the topological proof, we could again confirm the previous results with a simple syntactic re-check of TOrPEDO. • Step 4, With the purpose of satisfying property φ 2 , we produced a Revision 3 which included refinements of the Ste11 proposition to the value ⊥. Such change impacted clauses specified in the topological proof. As such, we had to re-execute the analysis phase and obtained, in conclusion, that also property φ 2 was satisfied by the third revised instance of the model. • Step 5. Finally, we also inspected the counterexample obtained by running TOrPEDO on the third revised model against φ 3 and, by changing the truth value of a number of propositions in the model we produced a Revision 4. We run the analysis phase and obtained a result also for the last property.
Results. This exemplifying iterative design process demonstrated that we were able to evaluate three properties on five different models (an initial one, plus four different revisions). We evaluated each of these properties by only running the analysis three times, while using a simple syntactic check twice. The topological proofs provide useful information to identify the parts of the models to be changed. They effectively enabled us to identify the portions of the model that influenced the satisfaction of the properties of interest. This information was useful for designing the model revisions. For example, in Step 2 we changed the value of a proposition. Since this proposition was not involved in any of the clauses of the topological proof, we could safely change its value. The re-check component was only executed for confirmation. Similarly, in Step 3 we applied another change to the model, this time by adding new transitions among given states. As for the previous step, our change did not involve any part of the model that was mentioned in the topological proof. Again, we verified our change by running the re-check component. In these two cases, the re-check phase confirmed that the revisions were compliant with the topological proofs and avoided re-executing the model checker.
Our experiment showed that the information contained in the topological proof was useful in our model design as it effectively guided us during the creation of the model revisions of a (small) model example from the genomic domain, related to Gene Regulatory Networks (GRNs). The answer to RQ4 is that the topological proofs and counterexamples provided by TOrPEDO effectively supported the development of a (P)KS representing a gene regulatory network.
Tao and Li [TL17] propose a theoretical solution to model repair: the problem of finding the minimum set of states in a KS which makes a formula satisfiable. However, the problem is different from the one addressed in this paper. Furthermore, the framework is only theoretical and based on complete systems.
Approaches were proposed in the literature to provide explanations by using different artifacts. For example, other works commonly use a different notion of witnesses: a path of the model that satisfies (or possibly satisfies) a property of interest [TG19, BCC + 99, HLSU02, Nam01, TGNB20]. In our work, we proposed topological proofs, a new type of witnesses that is significantly different from the one presented in the literature. Other works (e.g., [GC03b,SG03]) studied how to enrich counterexamples with additional information in a way that allows better understanding the property violation. Work has also been done to generate abstractions of the counterexamples that are easier to understand (e.g., [EMA10]). Alur et al. [AMT13] analyzed the problem of synthesizing a controller that satisfies a given specification. When the specification is not realizable, a counter-strategy is returned as a witness. Pencolé et al. [PSMTM17] analyzed model consistency, i.e., the problem of checking whether the system run-time behaviour is consistent with a formal specification. Bernasconi et al. [BMS + 17] proposed an approach that combines model checking and deductive proofs in a multi-valued context. The notion of topological proof proposed in this work is substantially different from the notion of deductive proof.
The works on vacuity checking (e.g., [FKSFV08,MS20,SDGC10]) are also related to our work. Specifically, a property φ is vacuously satisfied on a model M if it has a subformula φ that does not affect the satisfaction of φ in M . Other works (e.g., [PQ13, RLF + 13, SDGC10]) studied how to understand why a property is unsatisfiable. These problems are different from the one considered in this paper, where the goal is to provide a slice of the original model that preserves the (possible) satisfaction of the property.

Conclusions
We have proposed TOrPEDO, an integrated framework that supports the iterative creation of model revisions. The framework provides a guide for the designer who wishes to preserve slices of her model that contribute to satisfying fundamental requirements while other parts of the model are modified. For these purposes, the notion of topological proof has been formally and algorithmically described. This corresponds to a set of constraints that, if kept when changing the proposed model, ensure that the behavior of the model w.r.t. the property of interest is preserved. Our Lemmas and Theorems prove the soundness of our framework, i.e., how it preserves correctness in the case of PKS and LTL. The proposed framework can be used as a baseline for other FM frameworks, and can be extended by considering other modeling formalisms that can be mapped onto PKSs.
We presented two implementations of TOrPEDO, namely TOrPEDO-MUP and TOrPEDO-SMT. TOrPEDO-MUP is our initial implementation [MRB20] of TOrPEDO that uses PLTL-MUP to extract topological proofs. With the intent of providing practical and efficient support for flexible model design and wider adoption of our framework, in this work, we proposed TOrPEDO-SMT. TOrPEDO-SMT uses SMT techniques to extract topological proofs.
TOrPEDO was evaluated by showing the effectiveness of the analysis and re-check algorithms included in the framework. Results showed that proofs are smaller than the original models, and can be verified in most of the cases using a simple syntactic check. Additionally, we analyzed the efficiency of TOrPEDO and compared TOrPEDO-MUP with TOrPEDO-SMT. Our results show that TOrPEDO produces a topological proof within two hours for models with less than 40 states. Furthermore, our results show that TOrPEDO-SMT is more efficient than TOrPEDO-MUP. Note that, the analysis phase of TOrPEDO combines three-valued model checking and existing tools for the UCs computation. Therefore, its scalability improves as the performance of these frameworks enhances. Finally, we assessed how useful is TOrPEDO in supporting the evaluation of alternative design choices of (small) model instances in applied domains. Our results show that the topological proofs and counterexamples provided by TOrPEDO effectively supported the development of a model of a gene regulatory network.