Devil in the Landscapes: Inferring Epidemic Exposure Risks from Street View Imagery

Built environment supports all the daily activities and shapes our health. Leveraging informative street view imagery, previous research has established the profound correlation between the built environment and chronic, non-communicable diseases; however, predicting the exposure risk of infectious diseases remains largely unexplored. The person-to-person contacts and interactions contribute to the complexity of infectious disease, which is inherently different from non-communicable diseases. Besides, the complex relationships between street view imagery and epidemic exposure also hinder accurate predictions. To address these problems, we construct a regional mobility graph informed by the gravity model, based on which we propose a transmission-aware graph convolutional network (GCN) to capture disease transmission patterns arising from human mobility. Experiments show that the proposed model significantly outperforms baseline models by 8.54% in weighted F1, shedding light on a low-cost, scalable approach to assess epidemic exposure risks from street view imagery.


INTRODUCTION
With the rapid urbanization progress in the last century, more than 55% of people live in cities surrounded by the built environment that provides the setting for all human activities, such as buildings, roads, green spaces, etc..The intimacy between human and urban built environment makes it a critical environmental determinant of health [4], affecting both the physical and mental status of citizens.
With the recent development of big data processing and deep learning technology, street view imagery provides a powerful data source to assess the built environment with rich information and high scalability [7].In Figure 1, we showcase that house types, the number of vehicles, the design of green spaces are possible transmission-related features for predicting epidemic exposure risks [9].With publicly available street view imagery from map services or social media check-ins [8], we can predict the epidemic exposure risk in most parts of the world, even in low-and middle-income countries that may lack detailed census data.
However, accurately predicting epidemic exposure risk from street view imagery is challenging.First, infectious diseases are greatly influenced by the human mobility, which cannot be properly identified solely from street view imagery.Second, the distinct and complex transmission patterns arising from person-to-person contacts and interactions require different modeling approaches from traditional non-communicable diseases.
To overcome these challenges, we propose a novel model to identify epidemic exposure risks from street view imagery.Specifically, we construct a network of street view imagery to capture the regional transmission influence of infectious diseases, where we simulate the population flow inspired by Stouffer's law of population movement and Tobler's first law of geography.Based on the  proposed network, we design a transmission-aware graph convolutional network (GCN) emulating epidemiological process inspired by Susceptible-Infectious-Recovered (SIR) model [10].Through the proposed model, we predict the global map of epidemic exposure risk in a low-cost, scalable manner, enlightening the design methodology for creating an epidemiologically resilient living environment through the power of geographic information system (GIS).
The contributions of this study can be summarized as follows: • We construct a regional transmission network informed by the street view imagery and human mobility, based on which the spatial correlation of epidemic can be accurately captured.
• We propose a transmission-aware GCN model with epidemiological knowledge considering the distinct transmission patterns of infectious diseases.• We conduct extensive experiments to validate the effectiveness of the proposed model, which outperforms the best baseline by 8.54% in terms of weighted F1, 3.33% in weighted precision, and 4.93% in weighted recall.

PRELIMINARIES
The SIR model is a well-established epidemiological model that leverages the following ordinary differential equations (ODEs) to depict the dynamic of the epidemic: The above model divides the whole population  into four states: ,  ,  for susceptible, infectious and recovered people accordingly.
There are two learnable parameters:  is the infection rate, while  is the recovery rate.It assumes a second-order contact between susceptible and infectious people for disease transmission as  () (), and a first-order natural recovery process   ().Leveraging the calibrated SIR model, we can estimate the basic reproduction number  0 , which reflects the transmissibility of the target disease under specific urban scenarios as follows: In this study, we use  0 as an agent for epidemic exposure risk.

METHODS
We illustrate the framework of this study in Figure 2. We propose a transmission-aware GCN model, i.e., EpiGCN to infer the epidemic exposure risk through publicly available street view imagery.To capture the human mobility induced transmission of infectious diseases, we construct a regional mobility network, where the node feature represents geo-tagged imagery and the edge weight reflects the population flow simulated by the gravity model [11].Inspired by the computational process of SIR ODEs, we design a novel message passing function that integrates the epidemiological model with representation learning.

Mobility Network Construction
To capture the regional contact and transmission patterns invoked by human mobility, we first construct an MSOA level mobility network as shown in Figure 3. Specifically, we have the graph G = (V, E), where V is the set of MSOA and E is the set of mobility influences.
Leveraging any CV backbone model   , we extract the MSOA level imagery feature by aggregating all the image dense embeddings of the corresponding MSOA, which serves as the node feature   , ∀ ∈ V.
To depict the human mobility influence between MSOAs, we adopt the gravity model [11] to simulate regional population flows as the edge weight as follows: where  ★ is the population number for MSOA ★,   is the Euclidean distance between , .We set the empirical parameters , ,  according to [1].Eq.( 5) predicts the population flows as proportional to neighborhood population and inversely proportional to travel distance.The design of numerator is inspired by Stouffer's law of population movement that people are more attracted by regions with more social interaction opportunities.Besides, the denominator depicts Tobler's first law of geography that people tend to visit nearby places to reduce the travel cost.

SIR Message Passing Design
Given the node feature   of target node  and the neighborhood node set N  .First, we leverage three different linear layers to transform the node feature into susceptible (S), infectious (I), and recovered (R) embedding as follows: where  ★ is weight matrix,  ★ is the bias matrix, and  is the rectified linear unit (ReLU) activation function.
The essence of SIR model lies in the second-order transmission process of  () () and the first-order recovery process of  ().For the transmission process, we aggregate the I embeddings from neighborhood nodes according to the edge weights generated by the gravity model, which will be concatenated with local S embedding to depict the second-order interaction.It goes through a linear transformation to calculate the embedding of infections.For the recovery process, the only influence factor is the local I embedding.We use another linear transformation to depict the natural recovery process that happens in infectious people.The above process is shown in the following equations: where   ∈ R 2 × ,   ∈ R  × represent the linear transformation for the transmission and recovery process accordingly,  is the embedding size.
To get the epidemic exposure risk prediction, we further concatenate the local S, I, R embedding and use another linear transformation   ∈ R 3 × to capture the epidemic exposure risk as follows:

EXPERIMENTS 4.1 Setup of the Experiment
In this work, we study 6512 middle layer super output areas (MSOAs) in England, which are fine-grained census units with a mean population of 8236 and an average area of 19.5 km 2 .We collect the latest street view images from Google Map for each MSOA, where we uniformly sample 9 locations within the corresponding boundary [15].As a result, we get 215,759 images for the whole of England MSOAs, where 76.8% of them are captured after 2019.The street view images are available in 400 × 300, which will be randomly cropped into 224 × 224 before feeding into CV models.We download the MSOA level time series of COVID-19 cases in the second outbreak window (2020-09-01 to 2021-04-30) from the UK government.
In this study, we adopt the basic reproduction number  0 as an agent for epidemic exposure risk, which is a widely used metric to depict the severity of infectious diseases.Specifically, we calibrate the SIR model according to the COVID-19 time series, which provides a model-informed  0 for each MSOA.Furthermore, we categorize the extracted  0 in each MSOA according to the mean and standard deviation into three levels, which generates the epidemic exposure risk label  ∈ {0, 1, 2} for low risk, medium risk, and high risk accordingly.The distribution of labels is demonstrated in Table 1.We randomly split the dataset into training, validation, and test sets in a 6 : 2 : 2 ratio.
We implement the proposed EpiGCN in PyTorch, where we use ResNet18 initialized with ImageNet pre-trained weights as the CV backbone.Note that the whole architectures of the CV backbone are trainable.We adopt cross-entropy as the loss function.The implementation code of our model is available at https://github.com/0oshowero0/EpidemicGCN.
• BOF [12]: Bag of feature method leverages HOG and GIST to extract geo-tagged imagery, which follows a random forest classifier to generate predictions.• SceneParse [6]: SceneParse leverages the coverage ratio of each object to train an MLP for downstream tasks.• ResNet18 [5]: An end-to-end deep learning CV model that follows pyramid architecture.• ViT-B/32 [2]: An end-to-end deep learning CV model that follows isotropic architecture.

Figure 1 :
Figure 1: Illustration of epidemic exposure risk identification in Birmingham.In this study, we investigate the epidemic exposure risks for the whole England.

Figure 3 :
Figure 3: Illustration of the network construction process.The size of node represents the population, and the edge color represents the strength of population flow simulated by the gravity model.

Table 1 :
Distribution of epidemic exposure risk labels