Survey of Transient Execution Attacks and Their Mitigations Survey of Transient Execution Attacks and Their Mitigations

This article provides a survey of existing transient execution attacks from January 2018 to December 2020. We start by providing background on the micro-architectural features that lead to the attacks. We then define the transient execution attacks and summarize the phases and attack scenarios. We analyze the types of transient execution and covert channels leveraged by the transient execution attacks to show the root causes of these attacks. In the end, we discuss the mitigation strategies for the transient execution and covert channels. The contributions of this survey are the following:

• We summarize different attack scenarios and the security boundaries across which secrets are leaked in the different attacks.
  
• We provide a taxonomy of the existing transient execution attacks by analyzing the causes of transient execution that they leverage, and we propose metrics to compare the feasibility of using different transient execution types for attack.
  
• We summarize and categorize the existing and potential timing-based covert channels in micro-architecture states that can be used with transient execution attacks, and also propose metrics to compare these covert channels.
  
• We discuss the feasibility of the existing attacks based on the metrics we propose.
  
• We compare the different mitigation strategies that have been so far designed at the micro-architectural level in various publications.
  

Due to the data dependencies between instructions, the CPU pipeline sometimes has to stall until the dependencies are resolved. To reduce the stalls in the pipeline and to keep the pipeline full, many performance-improving optimizations have been proposed and implemented in today's commodity processors.

Out-of-Order Execution (OoO): In OoO, some of the younger instructions can be executed earlier than the older instructions (based on program order) if all the dependencies of the younger instructions are available. OoO helps to improve instruction-level parallelism. In OoO, the life cycle of instruction is fetch, dispatch, issue (a.k.a. execute), and retire (a.k.a. commit). The instruction is first fetched into the pipeline, and after decoding, micro-ops are dispatched. Once all the dependencies of the instruction are satisfied, an instruction (or micro-op) is issued for execution. OoO uses a reorder buffer (ROB) to hold the instructions (or micro-ops) and execution results and to retire each instruction (or micro-op) in the program order. Instructions (or micro-ops) are retired (committed) when they reach the head of ROB.

Speculative Execution: Speculative execution occurs when there is a control flow instruction for which the processor does not know the result yet. For example, a branch condition needs to be computed before the branch result (taken or not taken) can be obtained. To improve performance, processors often speculate the outcome of such instructions and begin to execute instructions down the speculated path. As the instructions execute down the mis-speculated path, they may access data that should otherwise not be accessible if the speculation was not allowed. Later, either speculation is confirmed to be correct or there is a mis-speculation, and instructions that began to execute down the mis-speculated path are squashed (cleared from the pipeline). After mis-speculation, the processor should clean up all architectural and micro-architectural states to create an illusion as if these instructions never executed. In addition to control flow, speculation may also happen in prefetching and value prediction (not used in commodity processors).

Delayed Fault Validation: Similar to speculation for control flow, processors speculate that there is no fault and continue execution even when there is a potential fault. Fault checking especially can be delayed, and some processors allow for instructions to continue executing, during which time the data (that should have been inaccessible due to the fault) can be accessed and micro-architecture states can be modified. If there is any fault, the processor then should clean up all the architectural and micro-architectural state changes.

Caching: One of the key optimizations for memory-related operations is caching. Modern processors have multiple levels of caches, typically L1, L2, and Last Level Cache (LLC). A cache hit occurs when a piece of data is found in a cache, and the processor does not need to go to the main memory to fetch the data. Based on the level of cache where data is found, it can be up to about $500\times$ faster to get data if it is a cache hit. Other cache-like structures in processors, such as translation lookaside buffer (TLB), can also offer performance improvement by caching data or metadata.

Sharing of Functional Units in Hyper-Threading within a Processor Core: To reduce area and power consumption, functional units are shared among hardware threads in hyper-threading. Today, most processors are built with two-thread Simultaneous Multi-Threading (SMT) configurations. In SMT, in the execution stage, the processor decides with instructions from which threads can execute based on the available execution units. It is expected that not all programs need all the units at the same time, so the sharing allows for good performance while reducing the need to duplicate all units among all SMT threads. However, sharing also creates situations when two processes try to use the same unit—there may be contention and difference in the timing of the execution.

Sharing of Resources among Cores in Multi-Core Processors: Similar to sharing within the processor core, many resources are shared among cores. These include caches, prefetchers, various buffers, memory controllers, directories, and other cache coherence-related structures, memory busses, I/O, and so forth. Rather than each core having a separate one of these units, they are shared to reduce hardware resources. However, the sharing, again, can create timing differences based on the different operations.

Covert channels are communication channels that are not originally designed for information transmission but can indirectly exfiltrate the information across security boundaries [109]. The entity sending the information is called the sender in covert channels.³ Execution of some instructions by the sender causes the state of the processor to be modified. The states can be internal processor states or physical states such as temperature or EM emanations. Finally, the receiver observes the state change to learn about a bit of information transmitted (or leaked) by the sender.

Timing-based covert channels, which are the focus of this survey, leverage the changes in the processor, which can be observed by measuring the execution time of some operations in software. The state modification can be the architectural state or the micro-architectural state. However, because by design the processors should not leak any information from the architectural state (e.g., two processes in different memory spaces cannot access each other's memory), the majority of the timing channels abuse the micro-architectural state changes, which cannot be directly read, but which can be observed through timing differences. These covert channels are the result of the various performance optimizations discussed in Section 2.1.

There are many security boundaries (between different privilege levels or security domains) in a typical processor, as shown in Figure 2. The goal of the attacker of the transient execution attacks is to learn information related to the victim's protected data across the different boundaries. In Figure 2, we categorize the possible privilege levels, or security domains, where the attack can originate and wherefrom it is trying to extract data as follows:

1. Across user-level applications: The attacker and the victim are two separate user applications, and the attacker process tries to learn the memory content of another process; e.g., [10] demonstrates how an attacker process learns the private key of a separate victim OpenSSH process.
   
2. User-level program attacking the kernel: The attacker runs in the user level and wants to read the privileged data of the kernel; e.g., [72] demonstrates an attack that allows an unprivileged application to dump kernel memory.
   
3. Virtual machine attacking another VM: The attacker and the victim reside in two different guest virtual machines (VMs); e.g., [10] shows it is possible for an attacker VM to learn the private key of the OpenSSH server in the victim VM.
   
4. Virtual machine attacking the hypervisor: The attacker is a guest OS and the victim is the host hypervisor; e.g., [64] demonstrates an attack against KVM that leaks the hypervisor's memory when the attacker has full control of the OS inside a VM.
   
5. Attacking the victim running inside an enclave: The victim runs inside a security domain protected by some hardware scheme, such as Intel SGX [25], XOM [69], Aegis [108], Bastion [19], Sanctum [26], and Keystone [66], and the attacker code runs outside of it; e.g., [21] demonstrates such an attack that retrieves secret from inside the SGX enclave.
   
6. Across security domains protected by software: The victim runs inside the security domain protected by some software scheme, such as sandboxes in JavaScript, and the attacker code runs outside of it, as shown in [64].
   

All of the security boundaries listed above are broken by one or more of the existing transient execution attacks. Details about each attack type will be discussed in Section 4.7.

As shown in Figure 1, we divide the transient execution attacks into three phases:

1. Setup Phase: The processor executes a set of instructions that modify the micro-architectural states such that it will later cause the transient execution of the desired code (called disclosure gadget) to occur in a manner predictable to the attacker. An example of modifying the micro-architectural states can be achieved by performing indirect jumps to a specific address to “train” the branch predictor. The setup can be done by the attacker running some setup code or triggering a setup gadget in the victim's code so that the micro-architectural state is set up as the attacker expects.
   
2. Transient Execution Phase: The transient execution is actually triggered in this phase. The piece of code that accesses⁴ and transmits the secret into the covert channel is called disclosure gadget, following the terminology in [112]. The desired disclosure gadget executes transiently in this phase due to the prior training in the setup phase. The instructions belonging to the disclosure gadget are eventually squashed, and the architectural states of the transient instructions are rolled back, but as many of the attacks show, the micro-architectural changes caused by the disclosure gadget remain in the majority of today's processors, so secret data can be later decoded from the covert channel. This phase can be executed either by the victim or by the attacker.
   
3. Decoding Phase: The attacker is able to recover the data via the covert channel by running the attacker's code or by triggering a decoding gadget in the victim's code and observing the behavior or result of the execution.
   

During an attack, the Setup Phase and the Transient Execution Phase cause the transient execution of the disclosure gadget to occur. Then, the Transient Execution Phase and the Decoding Phase leverage the covert channel to transmit data to the attacker. Thus, the Transient Execution Phase is critical for both accessing the secret and encoding it into a channel.

Each phase listed above can be performed by the attacker code or by the victim code, resulting in eight attack scenarios shown in Figure 3. When a phase is performed by the victim, the attacker is assumed to have the ability to trigger the victim to execute the disclosure gadget. We categorize the attacks based on who is executing transiently.

The first possible cause of transient execution is mis-speculation. Modern computer architectures make predictions to make full use of the pipeline to gain performance. When prediction is correct, the execution continues and the results of the predicted execution will be used. In this way, prediction boosts performance by executing instructions earlier. If the prediction is wrong, the code executed transiently down the incorrect (mis-predicted path) will be squashed.

1. Control Flow Prediction: Control flow prediction predicts the execution path that a program will follow. Branch prediction unit (BPU) stores the history of past branch directions and targets and then leverages the locality in the program control flow to make predictions for future branches. BPU predicts whether the branch is to be taken or not (i.e., branch direction) by a using pattern history table (PHT), and what is the target address (i.e., branch or indirect jump target) by using the branch target buffer (BTB) or return stack buffer (RSB). The implementation details of PHT, BTB, and RSB in Intel processors will be discussed in Section 4.9.1.
   
2. Address Speculation: Address speculation is a prediction of the address when the physical address is not fully available yet, e.g., whether two addresses are the same. It is used to improve performance in the memory system, e.g., STL forwarding in the load-store queue and line-fill buffer (LFB) in the cache use address speculation. The implementation details of STL and LFB in Intel processors will be discussed in Section 4.9.2.
   
3. Value Prediction: To further improve the performance, while the pipeline is waiting for the data to be loaded from the memory hierarchy on a cache miss, value prediction units have been designed to predict the data value and to continue the execution based on the prediction. While this is not known to be implemented in commercial architectures, value prediction had been proposed in the literature [70, 71].
   

The second possible cause of transient execution to occur is exceptions. If an instruction causes an exception, the handling of the exception is sometimes delayed until the instructing is retired, allowing code to (transiently) execute until the exception is handled. There are a number of causes of exceptions, such as a wrong permission bit (e.g., present bit, reserved bit) in Page Table Entry (PTE), and so forth. A list of all the exception types or permission bit violations is summarized in [16]. In addition, Xiao et al. developed a software framework to automatically explore the vulnerabilities using exceptions on a variety of Intel and AMD processors [127].

Sometimes the exceptions are suppressed due to another fault, e.g., nested exceptions. For example, when using transactional memory (Intel TSX [55]), if a problem occurs during the transaction, all the architectural states in the transaction will be rolled back by a transaction abort, suppressing the exception that occurred in the middle of the transaction [102, 115]. Another way is to put the instruction that would cause exception in a mis-predicted branch. In this survey, even if the exception is suppressed later, we categorize the attack to be due to exceptions.

Another cause of transient execution is (external) interrupts. If a peripheral device or a different core causes an interrupt, the processor stops executing the current program, saves the states, and transfers control to the interrupt handler. In one common implementation, when stoping execution, the oldest instruction in the ROB will finish execution, and all the rest of the instructions in the ROB will be squashed; the instructions that were executed after the oldest instruction (but end up being squashed) are executed transiently. After the interrupt is handled, the current program may continue the execution; i.e., the instructions that are squashed will be fetched into the pipeline again.

The fourth possible cause of transient execution is load-to-load reordering. Current x86 architectures use the total store order (TSO) memory model [105]. In TSO, all observable load and store reordering are not allowed except store-to-load reordering, where a load bypasses an older store of a different address. To prevent a load-to-load reordering, if a load has executed but not yet retired and the core receives a cache invalidation for the line read by the load, the pipeline will be squashed. Transient execution occurs between the instruction issue and when the load-to-load reordering is detected.

Not all transient execution can be leveraged in an attack, and Table 2 shows the causes of transient execution in existing attacks. (Mis-)speculation is leveraged in Spectre attacks, e.g., [64]. Address speculation is leveraged in MDS attacks [84, 102, 115] and LVI [114]. Exceptions of loads or stores are leveraged in Meltdown attacks [72], Foreshadow attacks [113, 123], LVI [114], and so forth. Other types of exceptions, interrupts, and load-to-load reordering are not considered to be exploitable. Because the instructions that get squashed due to exceptions, interrupts, and load-to-load reordering are legal to be resumed later on, no extra data is accessible to the attacker during the transient execution.

Table 2. Data Leaked by Different Transient Execution Attacks

indicates that the attack can leak the protected data; indicates that the attack cannot leak the data. *indicates all hardware components that cause the corresponding transient execution; we combine them in the same row because the data leaked in the attacks are the same. **Coh. Data is short for coherent data. Non-coh. Data is short for non-coherent data.

Example code of different variants is shown in Figure 4. The victim code should allow a potential mis-speculation or exception to happen. In Spectre V1 [64], to leverage PHT, a conditional branch exists in the victim code followed by the gadget. Even if the condition offset < arr1_len is not true, the disclosure gadget may still execute transiently. The disclosure gadget first fetches the secret at arr1[offset] and then conducts a memory access whose address (arr2[sec*c]) depends on the secret value that encodes the secret in the cache states. Similarly, in Spectre V2 [64] and V5 [65, 78], the victim code has an indirect jump (or a return from a function) that uses BTB (or RSB) for prediction of the execution path. In Spectre V4 [77], to use STL, the victim code has a store following a load having potential address speculation. In LVI [114], a load that triggers a page fault (accessing trusted_ptr) will forward non-coherent data in the store buffer, which is injected by a malicious store (*arg_copy = untrusted_ptr), and then the secret data addressed by the injected value (**untrusted_ptr) is leaked. In Meltdown [72], the attacker code makes an illegal load to cause an exception. In MDS attack [84, 102, 115], a faulty load (value=*(new_page)) forwards non-coherent data in the buffer.

If the attacker wants to launch a transient execution attack, the attacker should be able to cause transient execution of the disclosure gadget in a controlled manner. We propose the following metrics to evaluate the different causes of transient execution in an attack:

• Security Boundaries That Are Broken: This metric indicates the security boundaries that are broken during the transient execution attacks—this will be discussed in Section 4.7.
  
• Required Control of the Victim's Execution: This metric evaluates whether the attacker needs to control the execution of victim code—details will be discussed in Section 4.8.
  
• Required Level of Sharing: This metric evaluates how close the attacker should co-locate with the victim and whether the attacker should share memory space with the victim to trigger the transient execution in a controlled manner—details will be discussed in Section 4.9.
  
• Speculative Window Size: This metric indicates how many instructions can be executed transiently—the speculation window size will be discussed in more detail in Section 4.10.
  

As discussed in Section 3.1, the attacker's goal is to access the coherent or non-coherent data across the security boundaries in the system. Table 2 lists the type of data and the security boundaries across which the data can be leaked in the known transient execution attacks, assuming all the instructions in the disclosure gadget can execute transiently and the covert channel can transmit information to the attacker.

If the victim is executing transiently, the disclosure gadget can read any coherent data that the victim could access architecturally, even if the semantics of the victim code do not intend it to access the data [64]. Hence, in these attacks, the attacker can break the isolation between the victim and the attacker and learn data in the victim's domain. For example, the SWAPGS instruction is a privileged instruction that is usually executed after switching from user-mode to kernel-mode. If SWAPGS is executed transiently in the kernel-mode in the incorrect path, kernel data can be leaked [12]. When the victim is executing transiently, the attacker can also learn the non-coherent data (e.g., stale data) and also data that depends on non-coherent data (e.g., data in an address that depends on non-coherent data). For example, in Spectre V4 [77], stale data that contains the address of the secret data in the store buffer is forwarded to the younger instructions transiently, and the disclosure gadget accesses and transmits the secret data to the attacker. As another example, in LVI attack [114], the attacker injects malicious value through buffers, such as STL or LFB, causing a victim's transient execution that depends on a value controlled by the attacker and potentially leaks the value in an address controlled by the attacker.

If the attacker is executing transiently, transient execution allows the attacker to access illegal data directly. As shown in Table 2, the security boundaries that are broken depend on the causes of transient execution. In some processor implementations, even if a load causes an exception due to permission violation, the coherent data might still be propagated to the following instructions and learned by the attacker. For example, in Meltdown [72], privileged data is accessible transiently to an unprivileged user even if the privileged bit in the page table is set. In L1 terminal fault (L1TF) [123], secret data in the L1 cache is accessible transiently even if the present bit in the page table is not set. In Table 2, the attacks leveraging exceptions are categorized by the cause of the exception, e.g., page fault (PF), and the related permission bit. Non-coherent data present in the micro-architecture buffers, e.g., LFBs or store buffer (STB), can sometimes be accessed by the attacker in transient execution [84, 102, 115]. In addition, in CROSSTALK [96], a hardware buffer called the staging buffer is discovered. The staging buffer is used for some type of off-core reads; e.g., it is used by the RDRAND instruction that requests Digital Random Number Generator (DRNG), and the CPUID instruction that reads from Machine-Specific Registers (MSRs). The staging buffer is shared across cores, and thus, the CROSSTALK paper demonstrated a cross-core attack where the victim fetched some data from DRNG, and the attacker then learned the random number stored in the staging buffer during transient execution.

For the attacks leveraging mis-speculation, (mis-)training is an essential setup phase to steer the control flow to execute the desired disclosure gadget. The (mis-)training can be part of victim code, which is triggered by the attacker, as shown in Figure 3(b) and Table 1. In the example of Spectre V1, the attacker can first provide inputs to execute the setup gadget to train the branch predictor (i.e., PHT) to execute the target branch, because in this way the training code will always share the branch predictor with the attack code. In this case, the attacker should be able to control the execution of victim code. The (mis-)training code can also be a part of the attacker's code and run in parallel with the victim code, as shown in Figure 3(d), e.g., in Spectre V2. Then, it does not require control of the victim's execution for setup, but it requires the attacker's training thread and the victim's thread should be co-located to share the same prediction unit (e.g., BTB). Further, to share the same entry of the prediction unit, if the prediction unit is indexed by physical address, the attacker and the victim should also share the same memory space to share the entry, which will be discussed in the next subsection.

For the attacks that leverage exceptions, the instructions that follow the exception will be executed transiently, and thus, no mis-training is required, but the attacker needs to make sure the disclosure gadget is located in the code such that it is executed after the exception-causing instruction.

As shown in Table 1, in some scenarios, the setup phase and the transient execution phase are run by different parties, e.g., Figures 3(c) through 3(f), or in the attacker's different software threads, e.g., Figures 3(g) and 3(h). These cases require that the setup phase shares the same prediction unit (entry) with the transient execution phase. One common attack scenario is that the attacker mis-trains the prediction unit to lure the execution of the disclosure gadget of the victim, e.g., Figure 3(d). Hardware sharing can be the following:

• Same thread: The attacker and the victim (if both of them are executing) or the attacker's software threads (if only the attacker is executing) are running on the same logical core (hardware thread) in a time-sliced setting, and there might be context switches in between.
  
• Same core, different threads: The attacker and the victim (if both of them are executing) or the attacker's threads (if only the attacker is executing) are running on different logical cores (hardware threads) through SMT on the same physical core.
  
• Same chip, different cores: The attacker and the victim (if both of them are executing) or the attacker's threads (if only the attacker is executing) are on different CPU cores but are sharing LLC, memory bus, and other peripheral devices.
  
• Same motherboard, different chip: The attacker and the victim (if both of them are executing) or the attacker's threads (if only the attacker is executing) share memory bus and peripheral devices.
  

Some prediction units have multiple entries indexed by address or thread ID, and in that case, the attacker needs to share the same entry of the prediction unit with the victim during the setup. To share the same entry, the attacker needs to control the address to map to the same predictor entry as the victim. The address space can be one of the following:

• In the same address space: In this case, the attacker and the victim have the same virtual-to-physical address mapping.
  
• In different address spaces with shared memory: In this case, the attacker and the victim have different virtual-to-physical address mappings, but some of the attacker's pages and the victim's pages map to the same physical pages. This can be achieved by sharing dynamic libraries (e.g., libc).
  
• In different address spaces without shared memory: The attacker and the victim have different virtual to physical address mapping. Further, their physical addresses do not overlap.
  

In the following, we discuss the level of sharing required to trigger transient execution of disclosure gadget for an attack leveraging mis-speculation. In particular, the scenario depends on the implementation, and thus, we discuss each of the prediction units in Intel processors in detail.

4.9.1 Control Flow Prediction. To predict the branch direction, modern branch predictors use a hybrid mechanism [32, 57, 81, 83, 106]. One major component of the branch predictor is the PHT. Typically, a PHT entry is indexed based on some bits of the branch address, so a branch at a certain virtual address will always use the same entry in the PHT. In each entry of the PHT, a saturating counter stores the history of the prior branch results, which in turn is used to make future predictions.

To predict the branch targets, a BTB stores the previous target address of branches and jumps. Further, a return instruction is a special indirect branch that always jumps to the top of the stack. The BTB does not give a good prediction rate on return instructions, and thus, RSB has been introduced in commercial processors. The RSB stores $N$ most recent return addresses.

In Intel processors, the PHT and BTB⁵ are shared for all the processes running on the same physical core (same or different logical core in SMT). The RSB is dedicated to each logical core in the case of hyper-threading [78]. Table 3 shows whether the prediction unit can be trained when the training code and the victim are running in parallel in different settings. The results are implementation dependent and Table 3 shows the result from Intel processors.

Table 3. Level of Sharing and (Mis-)training the Prediction Unit on Intel Processors

	Prediction Unit
Ctrl Flow	PHT [36, 63]	f(virtual addr)	f(virtual addr)	–	–
	BTB [34, 64]	f(virtual addr)	f(virtual addr)$^{a}$	–	–
	RSB [78]	Not by address$^{b}$	–	–	–
Address	STL [56, 84]	f(physical addr)$^{c}$	–	–	–
	LFB [102, 115]	Not by address	Not by address	–	–
	Other$^{d}$
Value	No commercial implementation
“–” indicates the prediction unit is not possible to be trained under the corresponding sharing setting. Otherwise, the prediction unit can be trained and “f(virtual addr)” indicates the prediction unit is indexed by a function of the virtual address, “f(physical addr)” indicates the prediction unit is indexed by a function of the physical address, and “not by address” indicates the prediction unit is not indexed by addresses. $^{a}$Conflicting results are presented in different publications [34, 64]. $^{b}$Most OSs overwrite RSBs on context switches. $^{c}$STL is possible after context switch but not on SGX enclave exit. $^{d}$In [102], it is indicated that there could be other structures that forward data speculatively.

The prediction units sometimes have many entries, and the attacker should use the same entry as the victim for mis-training. The attacker and the victim will use the same entry only if they are using the same index. When the prediction unit is indexed by virtual address, the attacker can train the prediction unit from another address space using the same virtual address as the victim code. If only part of the virtual address is used as the index, which is shown as f(virtual addr) in Table 3, the attacker can even train with an aliased virtual address, which maps to the same entry of the prediction unit as the victim address. The RSB is not indexed by the address; rather, it overflows when many nested calls are made, and this creates conflicts when there are more than $N$ nested calls, and will cause mis-speculation.

To let an attack happen, there should be a large enough speculative window for the disclosure gadget to finish executing transiently, as shown in Figure 1. The speculative window size is the window from the time the transient execution starts (instruction fetch) to the time the pipeline is squashed. In attacks leveraging mis-speculation, the speculative window depends on the time the prediction is resolved. In a conditional branch, the time depends on the time to resolve the branch condition; in indirect jump, this depends on the time to obtain the target address; and in address speculation, this depends on the time to get the virtual and then the physical address. In [79], a tool called Speculator is proposed to reverse-engineer the micro-architecture using hardware performance counters. The results of Speculator show that the speculative window of branches that depend on uncached data is about 150 cycles on Intel Broadwell, about 300 cycles on Intel Skylake, and about 300 cycles on AMD Zen, and the speculative window of STL is about 55 cycles on Intel Broadwell. In attacks leveraging exceptions, the speculative window depends on the implementation of exceptions. To make the speculative window large enough for the disclosure gadget, the attacker can delay obtaining the result of the branch condition or the addresses by leveraging uncached loads from main memory, chains of dependent instructions, and so forth.

This survey focuses on covert channels that do not require physical presence and that only require the attacker's software (or software under the attacker's control) to be executing on the same system as the victim. Thus, we do not consider physical channels, such as power [39], EM field [80], acoustic signals [6, 40], and so forth. There are certain physical channels that can be accessed from software and not require physical presence, such as temperature [128]. However, thermal conduction is slow and the bandwidth is limited.

When considering channels that do not require a physical presence, in general, any sharing of hardware resources between users or programs can lead to a covert channel [119]. The receiver user or program can try to observe the states of the hardware through the execution time, the values of hardware performance counters (HPCs), or system behavior. The most commonly used observation by the receiver of the covert channels is the timing of execution. In today's processors, components are designed to achieve a better performance, and thus, the execution time contains information about whether a certain hardware unit is available during execution (e.g., port), whether the micro-architectural states are optimal for the code (e.g., cache hits or misses), and so forth. To observe the hardware states via timing, a timer is needed. In x86, the rdtscp instruction can be used to read a high-resolution timestamp counter of the CPU, and thus can be used to measure the latency of a chosen piece of code. When the rdtscp is not available, a counting thread can be used as a timer [103].

The receiver can also gain information from HPCs. HPCs have information about branch prediction, cache, TLB, and so forth, and have been used in covert channel attacks [36]. However, HPCs must be configured in kernel mode [27], and thus are not suitable for unprivileged attackers.

The receiver can further observe the state of the hardware by the system behaviors. In Prime+Abort attack [30], for example, Intel Transactional Synchronization Extensions (TSXs) [55] can be exploited to allow an attacker to receive an abort (call-back) if the victim process accessed a critical address.

In transient execution attacks, several covert channels can also be used in series. Here, we focus on the channels where the receiver can eventually decode data from the architectural states, e.g., values in register files. For example, in the Fetch+Bounce covert channel [99], first, the secret is encoded into the TLB states, which affect the STL forwarding, and then a cache Flush+Reload covert channel is used to observe the STL forwarding results. The first channel can only be observed by instructions in transient execution, and the states will be removed when the instruction retires. We only consider the second covert channel to be critical for transient execution attack because it allows the attacker to observe the secret from the architectural states.

We categorize the covert channels into volatile channels and persistent channels. In volatile channels, the sender and the receiver share the resource on the fly, and no states are changed; e.g., the two parties use a port or some logic concurrently. There is resource contention when the sender and the receiver communicate using this type of channel. In persistent channels, the sender changes the micro-architectural states, and the receiver can observe the state changes later, e.g., change of cache state. Although the states may be changed later, we call them persistent channels to differentiate from the volatile channels.

In a volatile covert channel, there is contention for hardware resources between the sender and the receiver on the fly, and thus, the two should run concurrently, for example, as two hyper-threads in SMT processors, or running concurrently on two different cores. Another scenario is that the sender and the receiver are two parts of code in the same software thread that their instructions are scheduled to execute concurrently due to OoO [37]. As shown in Figure 5, in a volatile channel, the receiver first measures the baseline execution time when the sender is not using the shared resource. Then, the sender causes contention on the shared resource or not, depending on the bit of the message to be sent, while the receiver continues to measure the execution time. If the execution time increases, the receiver knows the sender is using the shared resource at the moment.

Execution units, ports, and buses are shared between the hyper-threads running concurrently on the same physical core and can be used for covert channels [2, 10]. There is also a covert channel leveraging the contention in the floating-point division unit [37]. L1 cache ports are also shared among hyper-threads. In Intel processors, L1 cache is divided into banks, and each cache bank can only handle a single (or a limited number of) requests at a time. CacheBleed [137] leverages the contention L1 cache bank to build a covert channel. Later, Intel resolved the cache bank conflicts issue with the Haswell generation. However, MemJam [86] attack demonstrates that there is still a false dependency of memory read-after-write requests when the addresses are of the same L1 cache set and offset for newer generations of Intel processors. This false dependency can be used for a covert channel. As shown in Table 4, the covert channel in execution ports and L1 cache ports can lead to covert channels within the same thread when the sender and the receiver code are executed in parallel due to OoO and between hyper-threads in the SMT setting.

Table 4. Known Micro-architectural Covert Channels

indicates that the attack is possible to leak the protected data; indicates that the attack cannot leak the data. $^{a}$ Depending on the level of TLB used, the required time resolution varies. The biggest one is shown. $^{b}$ Shows the time resolution for covert channel using L1 cache. $^{c}$ Depending on the setup, the required time resolution varies. The biggest one is shown.

Memory bus serves memory requests to all the cores using the main memory. In [126], it is shown that the memory bus can act as a high-bandwidth covert channel medium, and covert channel attacks on various virtualized x86 systems are demonstrated.

In a persistent channel, the sender and the receiver share the same micro-architectural states, e.g., registers, caches, and so forth. Different from volatile covert channels, the state will be memorized in the system for a while. And the sender and the receiver do not have to execute concurrently. Depending on whether the state can only be used by one party or can be directly accessed by multiple parties, we classify the persistent channels into occupancy based and encode based, as shown in Figure 6.

The covert channel is used in the disclosure gadget to transfer the secret to be accessible to the attacker architecturally. The disclosure gadget usually contains two steps: (1) load the secret into a register and (2) encode the secret into a covert channel. As shown in Figure 7, the disclosure gadget code depends on the covert channel used. For covert channels in the memory hierarchy (e.g., cache side channel), it will consist of memory access whose address depends on the secret value. For AVX-based covert channels, the disclosure gadget encodes the secret by using (or not using) AVX instruction.

We propose the following metrics to compare different covert channels:

• Level of Sharing: This metric indicates how the sender and the receiver should co-locate. As shown in Table 4, some of the covert channels only exist when the sender and the receiver share the same physical core. Other attacks exist when the sender and the receiver share the same chip or even the same motherboard.
  
• Bandwidth: This metric measures how fast the channel is. The faster the channel, the faster the attacker can transfer the secret. Table 4 compared the bandwidth of different covert channels. Usually, the bandwidth is measured in a real system considering the noise from activities by other software and the operating system.
  
• Time Resolution of the Receiver: As shown in Figures 5 and 6, the receiver needs to measure and differentiate different states. For a timing channel, the time resolution of the receiver's clock decides whether the receiver can observe the difference between the sender sending 0 or 1. The last column of Table 4 shows the timing difference between states. Some channels, such as cache L1, require a very high-resolution clock to differentiate 5 cycles from 15 cycles, while the LLC covert channel only needs to differentiate 500 cycles from 800 cycles, and the receiver only needs a coarse-grained clock.
  
• Retention Time: This metric measures how long the channel can keep the secret. In some of the covert channels (volatile channels in Section 5.3), no state is changed, e.g., the channel leveraging port contention [2]. The retention time of such channels is zero, and the receiver must measure the channel concurrently when the sender is sending information. Other covert channels (persistent channels in Section 5.4) leverage state change in micro-architecture, and the retention time depends on how long the state will stay; for example, the AVX2 unit will be powered off after about 1 ms. If the receiver does not measure the state in time, he or she will obtain no information. For other states, such as register, cache, and so forth, the retention time depends on the usage of the unit and when the unit will be used by another user.
  

Table 4 lists different micro-architectural covert channels. The existence of a covert channel depends on whether the unit is shared in the attack setting. For example, AVX2 units, TLB, and the L1/L2 caches are shared among programs using the same physical core. Therefore, a covert channel can be built among hyper-threads and threads sharing a logical core in a time-sliced setting. The LLC, cache coherence states, and DRAM are shared among different cores on the chip, and therefore, a covert channel can be built between different cores or different chips.

Some covert channels may use more than one component listed in Table 4. For example, in the cache hierarchy, there could be multiple levels of caches shared among the sender and the receiver. In the Flush+Reload cache covert channel, the receiver can use the clflush instruction to flush a cache line from all the caches, and the sender may load the cache line into L1/L2 of that core or the shared LLC. If the sender and the receiver are in the same core, then the receiver will reload the data from L1. If the sender and the receiver are in different cores and only sharing the LLC, the receiver will reload the data from the LLC. Therefore, even with the same covert channel protocol, the location of the covert channel depends on the actual setting of the sender and the receiver.

As shown in Table 4, the channels in caches have relatively high bandwidth ($\sim$1 MBits/s), which allows the attacker to launch efficient attacks. Covert channels in AVX and TLB are slower but enough for practical attacks.

Attacks leveraging mis-speculation require the attacker to mis-train the prediction unit in the setup phase to let the victim execute gadgets speculatively (Section 3.3.3 and Section 4.8). To be able to mis-train, the attacker either needs to control part of the victim's execution to generate the desired history for prediction or needs to co-locate with the victim on the same core. MDS attacks also require the attacker and the victim to share the same address speculation unit. As shown in Table 3 on the level of sharing required for the setup phase, the prediction unit is shared only within a physical core, and for some prediction unit, they may not even be shared between hyper-threads. In practice, it is not trivial to co-locate the setup phase and the transient execution phase on the same core.

When a covert channel across processes is required, the sharing of hardware is needed (Table 1 on the attack scenarios and Section 5.7), which requires the co-location between the transient execution phase and the decoding phase. Furthermore, for a specific attack implementation, only one disclosure primitive is used, and the attack can be mitigated by blocking the covert channel.

Most of the existing studies focus on Intel processors, Table 6lists the known attacks on processors from different venders, such as AMD [4, 16], Arm [5, 16], and RISC-V [41]. As shown in the table, Spectre attack and its variants using branch prediction are found on all the platforms; this is because branch speculation is fundamental in modern processors. Other types of transient execution depend on the micro-architecture implementation of speculation units and show different results on different platforms.

Table 6. Known Transient Execution Attacks on Different Platforms

Cause of Transient Execution		Intel	AMD [4, 16]	Arm [5, 16]	RISC-V[41]
Control Flow	PHT (V1)
	BTB (V2)
	RSB (V5)
Address Speculation	STL (V4,MDS)
	LFB (MDS)
Exception	PF-US (V3)
	PF-P (L1TF)
	PF-RW (V1.2)
	NM (LazyFP)
	GP (V3a)
	Other
indicates that an attack of the type on the platform; indicates that there is no known attack.

The simplest mitigation is to stop any transient execution. However, it will come with a huge performance overhead; e.g., adding a fence after each branch to stop branch prediction causes 88% performance loss [132].

To limit the covert channels, one way is to isolate all the hardware across the sender and receiver, so any state changes caused by the sender will not be observable to the receiver. However, this is not always possible; e.g., in some attack types such as Meltdown, the attacker is both the sender and the receiver of the channel.

Another mitigation is to eliminate the sender of the covert channel in transient execution. For volatile covert channels, the mitigation is challenging. For permanent covert channels, there should not be speculative change to any micro-architectural states, or any micro-architectural state changes should be rolled back when the pipeline is squashed. Covert channels in memory systems, such as caches and TLBs, are most commonly used. Hence, most of the existing mitigations focus on cache and TLB side channels.

InvisiSpec [132] proposes the concept of “visibility point” of a load, which indicates the time when a load is safe to cause micro-architecture state changes that are visible to attackers. Before the visibility point, a load may be squashed and should not cause any micro-architecture state changes visible to the attackers. To reduce performance overhead, a “speculative buffer” is used to temporarily cache the load, without modifications in the local cache. After the “visibility point,” the data will be fetched into the cache. For cache coherency, a new coherency policy is designed such that the data will be validated when stale data is potentially fetched. The gem5 [11] simulation results show a 7.6% performance loss for the SPEC 2006 benchmark [52]. Similarly, SafeSpec [60] proposes to add “shadow buffers” to caches and TLBs, so that transient changes in the caches and TLBs do not happen.

In Muontrap [1], “filter cache” (L0 cache) is added to each physical thread to hold speculative data. The proposed filter cache only holds data that is in Shared state, so it will not change the timing of accessing other caches. If the data is in Modified or Exclusive state in another cache and shared state in L0 is not possible without state change in the other cache, the access will be delayed until it is at the head of ROB. The cache line will be written through to L1 when the corresponding instruction commits. Different from the buffers in InvisiSpec [132] and SafeSpec [60], the filter cache is a real cache that is cleared upon a context switch, syscall, or when the execution changes security boundaries (e.g., explicit flush when exiting sandbox) to ensure isolation between security boundaries. Muontrap results in a 4% slowdown for SPEC 2006.

CleanupSpec [97] proposes to use a combination of undoing the speculative changes and secure cache designs. When mis-speculation is detected and the pipeline is squashed, the changes to the L1 cache are rolled back. For tracking the line addresses of speculative changes, 1 Kbyte storage overhead is introduced. To prevent the cross-core or multi-thread covert channel, partitioned L1 with random replacement policy and randomized L2/LLC are used. Because only a small portion of transient executions result in mis-speculations, the method shows an average slowdown of 5.1%. Similarly, ReViCe [61] proposes an undoing approach by adding a victim cache that stores the cache lines replaced by speculative loads. With 32-entry victim cache for L1 and 64-entry victim cache for L2, the performance overhead is 2–6%.

ReversiSpec [125] proposes a comprehensive cache coherence protocol considering speculative cache accesses. The proposed cache coherence protocol interface includes three operations: (1) speculative load, (2) merge when a speculative load is safe, and (3) purge when a speculative load is squashed. Compared to InvisiSpec [132], the speculative buffer only stores data when the data is not in the cache, and thus, less data movement will occur when a load is safe (merge). Compared to CleanupSpec [97], purge is fast as not all the changes have propagated into cache. The performance overhead is 8.3%.

Moreover, accessing speculative loads that hit in L1 cache will not cause side effects (except LRU state updates) in the memory system. Therefore, only allowing speculative L1 hits can mitigate transient execution attacks using covert channels (other than LRU) in the memory system. In Selective Delay [98], to improve performance, for a speculative load that misses in L1, value prediction is used. The load will fetch from deeper layers in the memory hierarchy until the load is not speculative. In their solution, 11% performance overhead is shown.

Meanwhile, many secure cache architectures are proposed to use randomization to mitigate the cache covert channels in general (not only the transient execution attacks). For example, Random Fill cache [74] decouples the load and the data that is filled into the cache, and thus, the cache state will no longer reflect the sender's memory access pattern. Random Permutation (RP) cache [120], Newcache cache [75, 121], CEASER cache [95], and ScatterCache [124] randomize memory-to-cache-set mapping to mitigate contention-based occupancy-based covert channels in the cache. Non Deterministic cache [59] randomizes cache access delay and de-couples the relation between cache block access and cache access timing. Secure TLBs [29] are also proposed to mitigate covert channels in TLBs. But again, all the possible covert channels need to be mitigated to fully mitigate transient execution attacks. Further, Cyclone [48] and PerSpectron [85] propose micro-architecture designs to detect cache information leaks across security domains.

Another mitigation is to degrade the quality of the channel or even make the channel unusable for a practical attack. For example, many timing covert channels require the receiver to have a fine-grained clock to observe the channel (the second metric in Section 5.6). Limiting the receiver's observation will reduce the bandwidth or even mitigate the covert channel [94, 101]. Noise can also be added to the channel to reduce the bandwidth (the third metric in Section 5.6).

However, the above mitigations only cover covert channels in memory systems. To mitigate other covert channels, there are the following challenges: (1) identify all possible covert channels in micro-architecture, including future covert channels, and (2) mitigate each of the possible covert channels. Formal methods are required in this process. For example, information flow tracking, such as methods in [28, 140, 141], can be used to analyze the hardware components, where the data of transient execution could flow to. Then, it can be analyzed if each of the components could result in a permanent or transient covert channel.