Detecting Vulnerable Nodes in Urban Infrastructure Interdependent Network

Understanding and characterizing the vulnerability of urban infrastructures, which refers to the engineering facilities essential for the regular running of cities and that exist naturally in the form of networks, is of great value to us. Potential applications include protecting fragile facilities and designing robust topologies, etc. Due to the strong correlation between different topological characteristics and infrastructure vulnerability and their complicated evolution mechanisms, some heuristic and machine-assisted analysis fall short in addressing such a scenario. In this paper, we model the interdependent network as a heterogeneous graph and propose a system based on graph neural network with reinforcement learning, which can be trained on real-world data, to characterize the vulnerability of the city system accurately. The presented system leverages deep learning techniques to understand and analyze the heterogeneous graph, which enables us to capture the risk of cascade failure and discover vulnerable infrastructures of cities. Extensive experiments with various requests demonstrate not only the expressive power of our system but also transferring ability and necessity of the specific components.


INTRODUCTION
Urban infrastructure [6] refers to the engineering facilities essential for the regular running of cities, including electricity supply, transportation, and communication, etc.These urban infrastructures exist naturally in the form of networks, including electricity networks, road networks, communication networks, etc. Further, various infrastructure networks are coupled and interdependent, forming a typical system of interdependent networks [3,4].Moreover, cities confront the danger of natural catastrophes such as earthquakes, typhoons, and rainstorms, which often destroy functional units of different urban infrastructures.Due to the interplay between different infrastructures and their functional reliance, destruction often extends beyond the affected area, leading to widespread paralysis of urban infrastructure.This exacerbates the vulnerability of urban infrastructures [3,4].Urban infrastructures are essential for the proper functioning of various services in a city, including housing and production, and are closely tied to the well-being of its citizens.Therefore, gaining a profound understanding of the vulnerability of urban infrastructure holds significant value.However, understanding and characterizing the vulnerability of urban infrastructure is a difficult task.First, networks of different urban infrastructures exhibit diverse topological structures [9,10,28], which differ in terms of cluster structure, cycle structure, degree distribution, centrality, etc.The high-dimensional topological structure and the diverse topological features make it challenging for knowledge-based expert systems [18] or feature engineering methods [1] to thoroughly characterize it, which is the first challenge.Second, there is a strong correlation between these topological characteristics and infrastructure vulnerability [5,14].The states of different infrastructure networks have complicated evolution mechanisms.For example, the evolution of vehicle distribution on road networks is driven by the interaction between traffic control signals and driver actions, while the evolution of the distribution of power flow in electricity networks is driven by Ohm's law.These evolution mechanisms compound the functional dependencies of the infrastructure network to exacerbate the higher-order propagation effects of damage.How to model the complicated effect of evolution mechanisms of urban infrastructures is the second challenge.
In this paper, we focus on modeling the vulnerability of interdependent networks composed of diverse urban infrastructures using a data-driven approach.We develop a graph convolutional neural network, which can efficiently and exhaustively extract the topological features of each infrastructure network as well as the topological features of the interdependency between them, via optimizing the loss function of graph reconstruction.Further, we construct a simulator of the interdependent networks composed of urban infrastructures and develop a reinforcement learning (RL) module by interacting with it.This module deeply models the network evolution mechanism by using the influence of damaged functional units as a reward, which can accurately characterize the vulnerability of the system.
The contributions of this work can be summarized as follows: • We propose a graph convolutional neural network, which performs a coupled graph construction through semi-supervised embedding learning and can efficiently and exhaustively extract the topological features of each infrastructure network as well as the interdependency between them.• We construct a reinforcement learning (RL) module, which makes full use of diverse graph embeddings.It can discover a set of vulnerable functional units of infrastructures in the specified reward setting, thus accurately characterizing the vulnerability of the system.• Extensive experimental results show that our proposed model can effectively characterize the vulnerability of the system while interacting with an urban simulator.Specifically, the influence of the damaged functional units selected by our proposed algorithm significantly beat those of state-of-the-art algorithms.

PROBLEM STATEMENT
Starting from the basic interdependent network, we select two closely connected infrastructure networks -road network [7] and electricity network [29] from the city.The road network consists of a multitude of roads and intersections, representing edges and nodes, respectively.and the electricity network is a tree-like structure digraph composed of a large number of power stations at different levels, such as 220 kV, 110 kV, and 10 kV power stations.These two networks are connected by traffic lights located at intersections and low-level (10kV) power stations.In the road network, the state of traffic lights directly affects the connectivity of the road network.Specifically, if a traffic light stops working due to damage or power failure, roads it connects will become chaotic and even crowded, and more roads could be affected over time.As for the electricity network, it transmits electricity level by level to infrastructures of other networks.For instance, the failure of a 220 kV power station will cascade to 110 kV power stations, resulting in the shutdown of certain 10 kV power stations, ultimately impacting the functioning of traffic lights linked to those 10 kV power stations.Therefore, there are three states for each infrastructure: normal, damaged, and invalid, and "invalid" means that a node stops working because the nodes it connected are damaged or invalid.
To represent the interdependent network, we consider a coupled graph G = (V, E) that consists two different structural graphs: the road network G  = (V  , E  ) and the electricity network is the set of edges, where E ′ = {     , ∀  ∈ V  , ∀  ∈ V  } is the set of interdependent directed edges between power stations and traffic lights.A notation table is organized to make this paper easier to understand shown in Table 1.We expect to discover a set of nodes from the interdependent network as the vulnerable nodes, that once they are damaged, the coupled graph will be greatly affected, where "affected" depends on scene and purpose when making decisions.
For instance, if we pay more attention to the condition of road network, the number of crowded roads can serve as the primary indicator of "affected"; if we care more about the state of electricity network, the power decrease in the grid could be the main indicator of "affected".

THE PROPOSED SYSTEM
In the course of detecting the vulnerable nodes in the coupled graph, based on the strong representation ability of graph neural network (GNN) for graph-structured data and the ability of deep reinforcement learning (DRL) to solve sequential decision-making problem, we devise a new system to deal with this task.Figure 1 illustrates the architecture of our proposed system.

Coupled Graph Representation
We expect to take into account the direct and indirect information of the graph while choosing nodes, but the state of the graph can be very complex.Traditional methods [23,37] usually use fixed features to represent graphs, such as degree distributions, clustering coefficients, and so on.Due to the complex evolutionary mechanism of the coupled graph, these methods are not applicable to most coupled graphs and perform poorly when they characterize the graph's structural information.Based on graph neural networks, we leverage semi-supervised embedding learning [40] to obtain the coupled graph embedding that could extract the node features and graph topology in a low-dimensional embedding space.
3.1.1Graph Construction.Given an urban interdependent network consisting of an electricity network and a road network, we represent it as a coupled graph G = (V, E) according to the previous setting.We map each node  ∈ V to a initial d-dimensional vectors f  and define a node embedding matrix Note that the initial embeddings f   and f   are pre-trained from the respective graphs by the method described below, while the initial embeddings of them are randomly set.To improve its characterization ability, we use graph neural networks [13] to incorporate node attributes and capture the coupled graph structural information and other unseen data.On account of initial embeddings that have already contained the information of subgraphs, we are able to explore more information about the interdependent relationship in the coupled graph.

Graph Neural
Network.Graph G = (V, E) and initial embeddings of all nodes {f  , ∀ ∈ V} are provided as input.Let  denote the depth and h   ∈ R  ×1 denotes the embedding vector for node  at step , each node  ∈ V aggregates the embeddings of its immediate neighbor nodes, h   , ∀ ∈ N () , into a single vector h  N () ∈ R  ×1 , where N (•) stands for the set of neighboring nodes.Note that h 0  is defined as the input features f  .The aggregation process is shown as follows, where AGG is a summation function that aggregates information from local neighbors [22],  stands for the rectifier function ReLU, MEAN is the mean function, and W  1 ∈ R  × is a weight matrix to be learned at step  + 1.By defining the depth or iterations , we can get the final representation for all nodes at depth , z  ≡ h   , ∀ ∈ V , which contains -order neighbor nodes' information.

Semi-supervised Embedding
Learning.There are cascade relationships between infrastructures that are not connected directly in the urban network, and the topology of a network implicitly reveal the signal of similarity and correlation among nodes, which can be used as supervised signals to obtain the representation vectors.Therefore, the task can be treated as a link prediction [16] problem, which aims to estimate the likelihood of the edges between nodes, based on observed edges and attributes of nodes.
Then we take all of the edges in the graph as a positive set and sample several edges that do not exist as a negative set, which forms a positive graph G  and a negative graph G  , respectively.
To learn parameters of the aggregator function and weight matrices W  , ∀ ∈ {1, . . ., } via stochastic gradient descent, we firstly define a link prediction function as follows, which calculates the inner product of embedding vectors for two nodes on each existing edge as predicted weight, on given graph G.
Then we apply a margin loss function [30] to the node embeddings Z  as follows, where  is a constant parameter, the second term of the loss function performs 2 regularization where Θ stands for model parameters and  controls the penalty strength.For G  and G  , the weights simply represent the presence or absence of edges, and for coupled graph, G, weights are set to even indicate the type of the edges.While unknown edges are predicted by the known edges, we can obtain more superior embeddings for the interdependent network with cascading relationships, through semi-supervised learning [35].

Vulnerable Nodes Detecting
We obtain a coupled graph embedding matrix through a graph convolutional neural network, which can efficiently extract the topology features of the interdependent network.To make full use of this information, and find a set of vulnerable nodes in the network, we construct a reinforcement learning module.Reinforcement learning [31] is a sequential decision process where the agent is trained to take optimal actions for different scenarios of an environment.We formulate the RL problem as a Markov decision process (MDP), which is defined by a tuple (S, A, P, R, ) with a set of states S, a set of actions A, a reward function R (, ), the transition function P ( ′ | , ) and a discount factor , ,  ′ ∈ S,  ∈ A. P ( ′ | , ) governs the joint probability distribution  ( ′ | , ) of transitioning to new state  ′ after taking action  in state .Given a policy  , which fully determines the agent' behavior, the action-value function   (, ) is defined as the expected reward when starting from , taking action , and following the policy .In our proposed system, the action is to choose vulnerable nodes, the state is the residual coupled graph being analyzed after chosen nodes are damaged and the reward is the resulting weighted summation of cascade failure from the environment which represents the given interdependent network.
3.2.1 Framework.Given the couple graph G = (V, E) and the corresponding pre-trained node embeddings Z  ∈ R  × | V | as the input of our model, we expect to sequentially select  vulnerable nodes from the graph as output for a complete epoch.We use the node embedding matrix Z  to represent the universal state  of the current network (environment), and the initial state can be defined as the graph pooling, s 1 = MEAN(Z  ) ∈ R  ×1 .Then at each step , the agent selects a node   ∼  (• | s  ) as the action, where  is the policy.At the same time, next state s +1 is calculated by removing node   as follows, where is determined by the agent under the policy .The learning objective is to discover an optimal policy  * that can maximize the total reward for this whole trajectory:  ≥1     .Based on Bellman equation [31], the optimal action-value function   * (s, ) can be derived as: A common method of obtaining a new policy for an action-value function is to adopt -greedy policy [17], which means that the agent either chooses the node with the highest value with probability 1 −  or chooses a node randomly with probability .It has been proved that the agent following -greedy policy could obtain the maximum total rewards when   (, ) is optimal [32].

Value Network and Optimization.
Owing to the agent's action that choosing a node at each step is discrete, and the action space is extremely large we design our vulnerable nodes detecting model inspired by Deep Q-Learning Network (DQN) algorithm [26], which takes advantage of a convolution neural network to approximate the   (s, ).To make full use of the convolution neural network and structural information learned by the node embeddings Z  when choosing nodes, the action-value function   (s, ) are defined as follows, where  1 ∈ R 2 × and  2 ∈ R  ×2 are the optimized parameters,  stands for a rectifier function and [s  , . . ., is used for the -greedy selection of the next action   m, and the agent stores the transition tuple (s  ,   ,   , s +1 ) into an experience replay buffer, the size of which is .The parameters Θ are trained over tuples sampled uniformly from the reply buffer via stochastic gradient to minimize the mean-squared loss descent : (9) where Θ represent the parameters of the target net Q, which only updates after several iterations by copying Θ of evaluation net .This approach ensures a more stable and efficient training process.Since the graph is extracted from real data, we obtain the reward  by simulating the network changes after chosen nodes are damaged.To detect the vulnerable infrastructures in the road network, given the chosen node   , reward  is the weighted summation of the decreased power of the electric network and decreased value of connectivity of the road network.At step , the reward function R (  ) are defined as follows, ) where   and   are weight coefficients,  (  , G  ) is a function that can obtain the decreased power of given electricity network G  after power station    is damaged by Power Flow Calculation,  (  , G) is a function that returns a set of invalid traffic lights affected by damaged power station    and (  , G  ) is a function that can calculate the decreased value of connectivity of the road network.We can set a reasonable reward by adjusting the weight coefficients, to detect the ideal vulnerable nodes on the coupled graph.

EVALUATION
In this section, we aim to evaluate the proposed model with extensive answers to the following research questions (RQs).
• RQ1: Could our proposed method achieve the best performance compared with existing methods?• RQ2: What about the transferability of our model?Specifically, how about the model's performance when transferred into a new interdependent network?• RQ3: What about the effectiveness of each component in the proposed method?• RQ4: Why can our method find the vulnerable nodes in the graph?
What about the failure process in the interdependent network?

Experimental Settings
4.1.1Experimental environment.For the single network, we leverage the electricity network G  with 10,887 nodes and 11,438 edges, including different levels of power stations such as 550kV and 220kV, and infrastructure such as traffic lights.In addition, two different road networks-primary road G  with 1,035 nodes and 1,161 edges, and tertiary road G  with 4,825 nodes and 5,025 edges included, are also applied the model introduced before to choose vulnerable nodes, using metrics followed in Section 4.1.2.For the interdependent network with 15,712 nodes and 21,191 edges, representing the coupled electricity network and the tertiary road we attack nodes consisting of two graphs, importing the basic rules that once a power station fails, the corresponding traffic lights stop working.

Metrics.
We use the representative metrics to evaluate the selection performance of the proposed model and the existing method as follows.
• Electricity Network: To assess the impact on the electricity network, we calculate the current power after each node is damaged or disabled using flow calculation within the electricity network.• Road Network: For the road network, we use ratio of connectivity [11], which is defined as follows, where   is the th connected component in the current graph G, and   is the size of   .Another metric for the road network is the size of GCC, which refers to the giant connected component, calculated by: • Coupled Network For the coupled network, we use the reward defined as the weighted summation of power or connectivity decrease after each node is damaged in the electricity network or road topology correspondingly, to jointly evaluate the performance on two networks.

Baselines.
We compare our proposed method with the following competitive baselines.
• DE [19,37]: Here we represent the degree of node   as   .This method sorts the degree   of each node   in the graph and chooses the greatest ten nodes to be removed sequentially.
• CI [27]: The Collective Influence (CI) measure is defined as the product of the node's reduced degree and the sum of the reduced degrees of the nodes that are within a constant hop away from it.This method is calculated by CI() = (  − 1)  ∈ N () (  − 1), for each node, which describes the proportion of other nodes that can be reached from a given node, assuming the nodes with higher CI values play more crucial roles in networks.The CI method sequentially removes the node with the highest CI value and recalculates the CI values of the remaining nodes for the rest following operations.
• GDM [15]: Graph Dismantling with Machine Learning (GDM), utilizes a supervised learning approach to dismantle the entire network.To generate the training data and labels, we randomly sample nodes from the complete set and classify them into positive and negative sets.We then calculate the ratio of nodes belonging to the positive set for those that exist in both sets.The pre-trained node embeddings obtained from a graph neural network are used as inputs to a neural network, such as a Multilayer Perceptron (MLP), for the supervised learning process.The neural network computes outputs for all nodes, which are subsequently sorted in descending order to identify nodes for removal as a comparison to our method.

Overall Performance Comparison (RQ1)
We first present the overall performance comparison in Figure • Performance varies by different baselines.Based on observations, we have noticed distinct performance differences among various baselines when applied to single and coupled networks.
In the electricity network, CI identifies nodes as more vulnerable compared to DE due to the clustering distribution of power stations.Specifically, a 110kV station typically has only one 220kV upstream connection but multiple 10kV downstream connections.In contrast, DE performs better than CI in road networks, specifically primary and tertiary roads, due to the degree homogeneity, where road nodes typically have one incoming and one outgoing connection.Comparing the heuristic methods in the coupled network, CI still outperforms DE.We have established a cascade rule where a power station failure leads to the shutdown of traffic lights supported by that station, resulting in congestion and chaos.This demonstrates that power stations have a greater influence in this coupled network.Regarding machine-assisted analysis, the method works better than heuristic approaches in road networks because it requires fewer enabled nodes to cover a majority of nodes using a set of samples.However, in the electricity network and coupled network, sampling from such a large number of nodes struggles to cover even half of the total nodes.This poses a challenge due to the significant disparity between  the volume of testing data and the available training data, resulting in unsatisfactory performance.• Our method achieves the best performance.In comparison to heuristic methods and other learned machine approaches, our method consistently identifies critical nodes to minimize the current state and maximize the defined reward.By damaging fewer than ten nodes out of a total of 10,887 in the entire graph, our method reduces the electricity power to approximately 50% and even lower in the case of road networks.The power decrease achieved by our method is 50% lower than that of CI, the connectivity obtained is approximately 75% lower in primary roads and about 43% lower in tertiary roads compared to the bestperforming alternative methods.Additionally, the accumulated reward in our process is approximately 37% higher than that of the CI method.• Our method gets the result fast.Our method efficiently simulates the entire environment and rapidly identifies vulnerable nodes, yielding a stable and effective solution within approximately 500 epochs.• Cascade failure exists.Cascade failure does exist between the topology of interconnectedness between infrastructures.Without prior knowledge, our method could understand and characterize such a cascading rule driven by the data itself and model the complicated effect of evolution mechanisms of urban infrastructures.

Study on Transferring Ability (RQ2)
As mentioned before, one of the primary purposes is to characterize the vulnerability of the urban system to descend the influence of damaged functional units in cities.However, inaccuracy is always a property of real-world data and the infrastructures will also be continuously built and renovated.Due to the time and computation costs to re-train a new agent finding vulnerable nodes, we try to get the node embedding that responds to the real-world fast and accurately, and leverage the original parameters of value networks to find nodes and protect the current system, to study the transferring ability of our framework [33].The detailed setup of the transferability experiments is provided in Appendix.The results have been shown in Figure 3, figure (a)-(c) show the experimental results on the electricity network, primary road, and coupled network, respectively.CI can still find fragile nodes in the electricity and road network.Compared with baselines methods on the newgenerated topologies, our transferring system still works better than heuristic methods including DE and CI, obtaining about 46% lower power decrease and 75% lower connectivity than the best baseline correspondingly and reaching a plateau after a small set of nodes damaging.In the coupled network, the total reward obtained by our method is about 20 times the reward of DE and 3.4 times the reward of CI.In conclusion, our method has very strong transferring ability and robustness enabling us to avoid huge costs of time and computation.In addition, the improvement is meaningful and evident as urban infrastructures change rapidly in the real world nowadays.

Ablation Study (RQ3)
In reinforcement learning, the agent's objective is to maximize the expectation of the total reward by exploring the environment and taking appropriate actions.As reinforcement learning algorithms become more powerful, agents can handle increasingly complex environments.However, when dealing with heterogeneous graphs consisting of different types of nodes and edges, the explorationexploitation process for the agent can become challenging.Our hypothesis suggests that providing the agent with a confusing or randomly generated environment, specifically random-generated node embeddings, may result in a longer training process to find a stable solution, and the obtained results may be unsatisfactory.
Using the same datasets and metrics as described in Section 4.2, we compare the training duration and final results between our RL agent trained with pre-trained node embeddings generated by GNN and those generated randomly.The comparison is presented in Figure 4, following the same layout as Figure 2. As indicated in the dataset introduction, the performance of the GNN + RL approach outperforms the learning process in a random environment.Notably, the performance gap between the two approaches is more pronounced in the case of the coupled network compared to individual networks.Additionally, the ablation study on the two road networks shows a subtle difference since road structures are simpler and more limited, allowing for similar performance when the training process is extensive.However, in the interdependent network with multiple types of nodes and edges, learning valuable information from random representations becomes challenging.This results in poor performance during testing, where the total reward plateaus and the metrics of the two interacting single networks show a downward trend.
In conclusion, we have confirmed the validity of our assumption.The inclusion of the GNN component in our framework is essential and significantly enhances the performance of the agent when dealing with graph environments, particularly heterogeneous graphs.

Case Study (RQ4)
Figure 5 visualizes the effects of a series of node damages, displaying three sets of images representing the pre-stage, middle stage, and post-stage of identifying vulnerable nodes in the interdependent network [39].In the case of the electricity network (a)-(c), direct destruction is denoted by green markers, while yellow markers represent indirect destruction.For the road network (d)-(f), light green indicates direct road collapse, magenta represents roads affected by ruined or non-functioning power stations, and red denotes roads separated from the largest connected component after the series of damages.
The impact of node selection in the electricity network is evidently more pronounced in terms of both the extent of influence and power decrease compared to the coupled network.For instance, damaging two nodes in the single electricity network reduces the power to approximately 6.59, whereas in the coupled network, the power remains above 79 even after the same number of nodes are damaged.However, when considering the connectivity in the road network, the topological changes undergo a significant shift as nodes in the electricity network are damaged.As the damage continues, the coverage area of normally operating power stations and the largest connected component become increasingly limited.This demonstrates the existence of cascade failure and validates our framework's ability to accurately characterize the vulnerability of urban infrastructures.Moreover, the number of directly collapsed roads is considerably lower compared to those affected by the electricity network and subsequently isolated.This finding aligns with our model's performance, which shows that the electricity network has a greater influence.

RELATED WORK 5.1 Data science in urban infrastructure
Urban infrastructure, including road networks, communication networks, electricity networks, etc., is one of the most important elements in the city, and there are many applications for datadriven urban computing research.The early efforts mainly focus on the modeling or prediction of urban infrastructure.For road networks, the existing works pay a lot of attention to the problem of road traffic forcasting [8], road traffic analaysis [25], travel time prediction [36], or using the road network to predict other elements in the city such as air quality [38].That is, the existing works are still concentrated on one specific kind of infrastructure network, ignoring the joint modeling of the interdependence relations among different networks, which we aim to address in this work.
Despite the early efforts, the resilience and vulnerability of urban infrastructure are less explored in the literature.In this work, we approach the problem of finding the critical node in the interdependent infrastructure network, with extensive applications in the real world.

Data-driven optimization on graph
Graphs widely exist in plenty of domains, such as social networks, user-item graphs in recommender systems, road networks, etc.The ubiquity of graphs makes the optimization task of graphs an important problem in many real-world applications.The typical optimization problems include the traveling salesman problem (TSP) [20], maximum independent set (MIS) [34], maximum cut (MaxCut) [12], etc.However, despite their importance and wide applications, these optimization problems are always NP-hard, leading to high-cost time complexity that is exponentially related to the graph size.The traditional methods to address it are based on heuristics guided by real-world experience from human experts.However, although heuristic methods may not perform badly, they still suffer from limitations, which can be resolved by data-driven machine learning models [2].First, the heuristic methods highly rely on problem-specific designs, which will be challenging for new scenarios.Second, the data-driven methods can extract useful features or prediction signals that may be implicit and hard to explain, making them less likely to be proposed by human experts.Khalil et al. [21] proposed to learn graph states with graph neural networks and greedily generate actions until the criteria are met, supporting the data-driven solutions for Minimum Vertex Cover, Maximum  Cut, and Traveling Salesman problems.Li et al. [24] propose to use a graph convolutional network to predict whether a given vertex belongs to the optimal set or not, converting the original problem to a supervised-learning task.The authors further combine the tree search method with the neural network module to ensure the exploration ability of the approach.Different from the existing works which only consider small graphs, in this work, we approach a real-world problem in the urban infrastructure with far larger graph and heterogeneous relations, along with the complex environment.

CONCLUSION
Addressing the vulnerability of urban infrastructure interdependent network, we propose a data-driven framework using graph neural network and reinforcement learning to detect vulnerable nodes in urban infrastructure networks.Requiring no prior expert knowledge but just the connected relationships between different types of nodes in the real world, it trains on the complicated network quite efficiently.Thanks to the transferring ability and robustness of our framework, we can ignore some slight perturbations in our data and trust our model's performance in terms of effectiveness.Besides, our framework also illustrates the necessity of an explicit representation for the complicated network.Our framework captures the risk of cascade failure and discovers vulnerable nodes in real-world systems, which turns out to be dramatically useful in protecting fragile nodes in urban infrastructure and designing more robust topologies for cities.

A APPENDIX A.1 Setting for Transferablity Experiment
We create mask graphs to represent graphs with slight perturbations.We maintain the same number of nodes but allow for changes in edges, such as deletion and addition, in the electricity network, primary road network, and coupled network.We then retrain the new node embeddings using Algorithm 1 and feed them into the value network of our primary agent to calculate the corresponding metrics introduced in Section 4.1.2during the node damaging process.Finally, we compare the results with the baseline, including CI and DE, on the newly generated topologies.

Figure 1 :
Figure 1: Illustration of our proposed system.

Figure 2 :
Figure 2: Performance of different methods on different networks.

Figure 3 :
Figure 3: Transferring ability for different Networks.

2 .
In figure (a)-(c) we demonstrate the testing results on single networks including one electricity network and two road networks.In figure (d)-(e) we demonstrate the testing results on coupled network consisting of electricity network and tertiary road network.Based on the results, we have the following observations.
(a) Power Decrease of Electricity Network (b) ANC Decrease of Primary Road (c) ANC Decrease of Tertiary Road (d) Reward Increase for Bigraph (e) Power Decrease in Bigraph's Electricity (f) ANC Decrease in Bigraph's Road
(a) Pre-stage of Electricity Network (b) Mid-stage of Electricity Network (c) Post-stage of Electricity Network (d) Pre-stage of Road Network (e) Mid-stage of Road Network (f) Post-stage of Road Network

Figure 5 :
Figure 5: Process of finding vulnerable nodes in interdependent network

Table 1 :
Notations.Coupled graph G, where V is the set of nodes and E is the set of edgesG  = (V  , E  ) Electric network G  ,where V  is the set of power stations and E  is the set of wires  (  , G  ) Function that can calculate the decreased power of G  after   is damaged  (  , G) Function that returns a set of invalid traffic lights affected by damaged   (  , G  ) Function that can calculate the decreased value of connectivity of G  after   is damaged   ,   Weight coefficients of G  and G