On the Detection of Image-Scaling Attacks in Machine Learning

Image scaling is an integral part of machine learning and computer vision systems. Unfortunately, this preprocessing step is vulnerable to so-called image-scaling attacks where an attacker makes unnoticeable changes to an image so that it becomes a new image after scaling. This opens up new ways for attackers to control the prediction or to improve poisoning and backdoor attacks. While effective techniques exist to prevent scaling attacks, their detection has not been rigorously studied yet. Consequently, it is currently not possible to reliably spot these attacks in practice. This paper presents the first in-depth systematization and analysis of detection methods for image-scaling attacks. We identify two general detection paradigms and derive novel methods from them that are simple in design yet significantly outperform previous work. We demonstrate the efficacy of these methods in a comprehensive evaluation with all major learning platforms and scaling algorithms. First, we show that image-scaling attacks modifying the entire scaled image can be reliably detected even under an adaptive adversary. Second, we find that our methods provide strong detection performance even if only minor parts of the image are manipulated. As a result, we can introduce a novel protection layer against image-scaling attacks.


INTRODUCTION
Image scaling is a ubiquitous preprocessing step in many machine learning and computer vision systems.Before an image is fed to a learning model for inference, it is usually scaled down to fixed dimensions.For example, the popular neural networks VGG19 [22] and ResNet [8] for object recognition expect fixed inputs of 224×224 pixels.While an extensive body of research has explored vulnerabilities in learning models [1,14], the attack surface of preprocessing has received little attention so far.An exception is recent work on image-scaling attacks [17,24], a novel class of attacks that enable an adversary to tamper with the result of the scaling process (see Figure 1).These attacks exploit that most scaling algorithms process only a minor fraction of the pixels in an image, so that a few perturbations allow for full control of its scaled version [17].
In contrast to other security threats to machine learning, imagescaling attacks are agnostic to the employed learning models.Successful attacks only require knowledge about the scaling algorithm and the target dimensions.Compared to the parameters of a neural network, these details are limited in the number of possible configurations and can also be inferred through remote queries to the model [24].As a result, image-scaling attacks pose a notable threat to practical systems: They enable misleading classifiers without access to the learning model and allow hiding backdoor triggers or poisoning attacks in training data [15].Figure 1 illustrates both cases.In the top row, the adversary misleads the classification by changing the entire image during scaling.In the bottom row, the adversary induces local changes in the lower left corner of the scaled image (black square).If this modification is performed on training data, it allows concealing an otherwise noticeable backdoor trigger.Hence, there is a need for effective safeguards that complement existing security mechanisms for machine learning.
Two defense strategies have been explored for addressing this threat: prevention and detection.In the first case, robust scaling algorithms or specific defense filters are applied to input images to prevent attacks.While these defenses are provably effective, as demonstrated by Quiring et al. [17], they can only prevent but not detect attacks.However, detection can be necessary in various cases: First, we can spot on-going attacks.This allows scanning image collections for attacks or identifying the adversary.Second, robust algorithms like area scaling are slower compared to vulnerable algorithms, so that a one-time check might be preferred in real-time settings.Third, detection allows protecting proprietary learning systems where components cannot be changed.

Global change Local change
Frequency analysis The detection of image-scaling attacks, however, has not been rigorously studied yet.In this paper, we address this research gap and present the first in-depth systematization and analysis of detection methods for image-scaling attacks.We identify two general paradigms that underlie existing detection approaches: frequency analysis and spatial analysis.In the first paradigm, the detection is based on searching for conspicuous traces in the frequency spectrum.In the second paradigm, the pixels are analyzed, either by leveraging the adversarial modification or the remaining clean pixels for further analysis.

Spatial analysis
We derive novel detection methods for these paradigms that are simple yet effective in spotting image-scaling attacks.Our methods significantly outperform previous approaches, as we can reduce heuristic elements and precisely pinpoint the detectable characteristics of image-scaling attacks.
We demonstrate the efficacy of our methods in a comprehensive evaluation.In contrast to prior work, we study the detection performance with a diverse evaluation setup, including all major learning platforms, scaling algorithms, a large dataset, static & adaptive adversaries, and different attack scenarios such as global & local changes.If the entire scaled image is modified, both paradigms allow a detection rate between 99% and 100%.Our frequency approaches are particularly effective with a perfect detection rate.In the challenging scenario where only a local area is manipulated, previous methods fail while the newly derived approaches remain effective.The frequency paradigm enables a detection rate of 90%.The spatial paradigm allows detecting at least three out of four attacks.Under an adaptive attacker, however, the paradigms' effectiveness flips: The frequency paradigm-the strongest in the static attack-does not withstand an adaptive attack while the spatial analysis remains robust.As a result, we conclude that both paradigms should be used in combination to complement each other.
Contributions.In summary, our contributions are as follows: (1) Systematization of detection.We present the first systematic analysis of detection methods for image-scaling attacks where we identify two general paradigms.
(2) Novel approaches to detection.Based on our analysis, we derive novel detection approaches that significantly outperform all approaches from previous work.
(3) Comprehensive evaluation.We empirically investigate the performance of all detection approaches in a comprehensive evaluation that covers all major learning platforms and scaling algorithms.
(4) Different attack models.Moreover, we carefully examine the limits of all detection approaches by experimenting with different attack scenarios, such as global & local changes, and with both static & adaptive attackers.
To foster further research in this area, our code is publicly available at https://github.com/EQuiw/2023-detection-scalingattacks.
Roadmap.We introduce image-scaling attacks in Section 2. The detection paradigms with our new methods are presented in Section 3 and the evaluation of their performance is given in Section 4.
Adaptive attacks are evaluated separately in Section 5. Finally, Section 6 discusses related work and Section 7 concludes the paper.

BACKGROUND
We start by introducing the background on image-scaling attacks before presenting our novel detection approaches.

Preprocessing in Machine Learning
When solving tasks of computer vision, image data is typically normalized and preprocessed before features are extracted and learning models are applied.In particular, image scaling is a widely used preprocessing step to bring images to a normalized form.Most learning algorithms expect a fixed input size and thus images with different or larger dimensions need to be scaled.For example, the deep neural networks VGG19 [22] and ResNet [8] require inputs of 224 × 224 pixels.Due to this frequent preprocessing, major machinelearning frameworks directly integrate different scaling algorithms.For instance, Caffe employs the image processing library OpenCV, PyTorch uses the library Pillow, and TensorFlow has its own implementation called tf.image.Following prior work [24], we thus focus our analysis on the libraries OpenCV, Pillow, and tf.image with their respective implementations of scaling algorithms.

Image-Scaling Attacks
Interestingly, the downscaling of images leads to a considerable attack surface in learning-based systems.By carefully modifying particular pixels, it becomes possible to control the output of the scaling algorithms and thus to change the content of the scaled image.In the following, we recap the current state-of-the-art of these image-scaling attacks [17,24].Figure 2 exemplifies the principle of the attack.Given a source image , the adversary tries to find a minimal perturbation Δ, so that the downscaling of the modified image  = ( +Δ) produces an output image, scale(), that matches the adversary's target image  .This attack can be modeled as a quadratic optimization problem: where the interval R is the allowed pixel range, as for example, R = [0, 255] for 8-bit images.
To deceive the victim, a successful attack has to fulfill two objectives: First, the scaled image, scale() = scale( + Δ), needs to match the target image  , i.e., scale() ∼  .Second, the attack image should be indistinguishable from the source image, i.e.,  ∼ .As a result, the adversary gets an attack image  that looks identical to the source  but changes to the target  after downscaling.This attack is independent of the training data, extracted features, and employed learning model.The adversary only needs to know the used scaling algorithm and the target size of the scaled image.In practice, this knowledge might not be difficult to obtain.Publicly available deep neural networks are often re-used through transfer learning.Moreover, the number of possible scaling configurations in the libraries is limited.In some settings, the knowledge about the algorithm is not even required.OpenCV and TensorFlow, for example, are effectively using nearest scaling when the scaling factor is an (uneven) integer [17].This simplifies an attack if the adversary can choose the size of .Finally, even without any knowledge, the scaling setup can be deduced through black-box queries [24].
Threat Scenario.By attacking the preprocessing, the input is manipulated at the very beginning of the learning pipeline.An adversary can thus efficiently control any subsequent steps, which enables and simplifies different attack strategies.First, during training time, the attacker can conceal poisoning and backdoor attacks: The training-data modifications become visible only after downscaling [15].This alleviates the shortcoming of attacks that leave visible artifacts in images [7,21,25].The combination is useful, for example, if backdoors are used in the physical world.At training time, scaling attacks hide the trigger that can be later activated in the physical world.Second, during test time, the adversary can control the predictions of a learning model-without modifying the training data or model.The adversary uses the scaling attack, so that the downscaling leads to an image of the target class.
Note that scaling attacks allow a similar attack as adversarial examples by causing a misclassification.However, they do not depend on the learning model or features, since the scaling stage produces a perfect image of the target class.Scaling attacks would succeed even if neural networks were robust against adversarial examples.
Attacks can be realized with varying degrees of modification.We differentiate two scenarios to understand the detection capabilities: • Global modification.The adversary chooses a target image with an arbitrary, unrelated content.The input  and target  have no relation to each other.This is the most severe attack, as any target class can be chosen.
• Local modification.The source and target image are identical, except for a limited area.We study backdoors as example for this category where only a small trigger is added (see Figure 9).A detection is challenging due to the small changes.
When studying the global scenario, we also consider an overlay scenario where the scaling attack only partially creates the target image  .Here, we blend  into the downscaled source image to create a novel target image.This scenario allows us to study an attacker who does not fully embed the target image into the scaling output.
Root Cause.Image-scaling attacks are possible because scaling algorithms do not process all pixels equally [17].Depending on the algorithm and scaling ratio, many pixels in  have limited or even no impact on the scaled output.An adversary can therefore only modify those pixels that are considered during scaling.The resulting sparse noise is visually imperceptible, yet controls the entire output of the scaling process (see Figure 3).In this way, both attack objectives, scale() ∼  and  ∼ , are fulfilled.We build our detection defenses on this understanding of scaling attacks.
Prevention Defenses.We also recap defenses that prevent imagescaling attacks.Prior work has studied two concepts: First, we can apply a robust scaling algorithm that considers all pixels with equal contribution.This directly addresses the root cause of the attack, since no pixels are ignored anymore.A possible scaling algorithm is area scaling that is implemented in several image processing libraries.Second, the defender can use an image-reconstruction method to repair the modified pixels from their neighborhood.To circumvent this defense, an adversary would have to modify the neighborhood as well, which leads to visible modifications.We refer the reader to the paper by Quiring et al. [17] which examines both concepts and their robustness.
While prevention mechanisms do not interfere with the typical machine-learning workflow, they have a clear disadvantage: The mechanisms cannot be applied to find out that an attack is going on, that is, the data is manipulated.However, this can be necessary to ensure that we can trust a specific dataset, for instance, if we use publicly available images or release our image database.Hence, there is a need to study effective detection methods as complementary approach to existing prevention concepts.

DETECTION SYSTEMATIZATION
The detection of image-scaling attacks has received little focus so far.While some heuristics have been proposed [10,24], we still lack a general understanding on how the attacks can be effectively characterized and detected.To fill this gap, we systematize existing work by identifying two general detection paradigms.For each paradigm, we present the basic principle of detection and introduce own realizations that alleviate shortcomings of existing heuristics.Table 1 shows an overview of all considered detection methods.

Paradigm: Frequency Analysis
In this paradigm, the detection builds on analyzing the frequency spectrum of images.This is a common procedure in computer vision and multimedia security [20].In this frequency representation, periodical patterns become evident that are not detectable in the spatial image domain (pixel domain) [11].In general, a frequency spectrum is an equivalent representation of an image that describes the pixels by a sum of waves oscillating at different frequencies [23].In this work, we use the 2D discrete Fourier transform (DFT) and work on the centered log-scaled magnitude spectrum (as visible e.g. in Figure 4a).Intuitively, each coefficient in this magnitude spectrum shows the impact of a particular frequency for the image.The log-scaling emphasizes smaller values.The middle of the magnitude spectrum corresponds to low frequencies while higher frequencies are located towards the corners.
As the root-cause analysis of image-scaling attacks in Section 2 highlights, the adversary injects pixels from the target image  into  in a periodic distance.This periodicity stems from the sampling process of image scaling and provides a strong indicator of image-scaling attacks.Hence, a modified image has unique, periodic peaks in its frequency spectrum.As an example, let us analyze the running example in Figure 2. The modification is not visible in the spatial domain.However, if we analyze the attack image in the frequency domain-as visible in Figure 4b-one can clearly see unusual frequency peaks.
Proposed Detection.For our analysis, we take inspiration from multimedia forensics [3,20].We start by noting that the defender exactly knows the potentially modified pixels in the spatial domain.This allows an exact computation of the expected peak locations in the frequency spectrum.As a result, we gain the advantage of differentiating adversarial peaks created by scaling attacks from benign peaks that images can naturally have at other locations in the spectrum, reducing the chances to detect benign peaks accidentally.
In particular, our detection approach proceeds as follows: Let (, ) be the height and width of the source image and ( ′ ,  ′ ) the height and width of the scaled output image.The vertical scaling ratio is given as   =   ′ while the horizontal scaling ratio is   =   ′ .The constants   and   are the index of the spectrum's middle.In the case of a scaling attack, the following binary function Γ ∈ {0, 1} × shows at which frequency coefficient a peak occurs: with In other words, we expect to observe peaks around each  1  ′ -th and  2  ′ -th position of the frequency spectrum if an image is manipulated by an image-scaling attack.In Appendix A, we derive Equation 2. To provide some intuition, Figure 4c marks the expected peaks on the frequency spectrum of our running example by using Equation 2. One can see that the observed and expected peaks match exactly.Equipped with the ability to predict the expected peaks, we propose two detection strategies.
Peak Spectrum.We extract the frequencies in a square window centered around all expected peak locations.The window's half length is .We omit the center (  ,   ) being present in any spectrum.Figure 5a illustrates the resulting windows.Next, we average all window values and calculate the percentile rank of this value relative to the whole frequency spectrum.This sets the peak frequencies into relation to the entire frequency spectrum.In case of an attack, the peak frequencies outshine the entire spectrum, so that attack images get higher percentile ranks than benign images.
Peak Distance.We divide the spectrum into excerpts for each peak as Figure 5b visualizes.We discard the center of the spectrum again.For each excerpt, we extract the maximum peak and calculate the distance between this peak and the expected peak.We then average all measured distances.In case of an attack, this average is expected to be small.Previous Approaches.Kim et al. [10] propose a frequency analysis, named CSP, where the underlying idea is to count peaks at arbitrary positions in the spectrum.Based on their evaluation, they assume an attack if the spectrum contains more than one peak.Our evaluation shows that this approach is ineffective.Benign images can naturally have peaks, too.We show two such examples from ImageNet in Appendix B. To test if the method works without the fixed threshold of one, we evaluate an own adjustment, named CSP-improved, where we derive the threshold from the data.Still, this adjustment cannot compete with our methods.The defender should rather use the advantage of having precise knowledge about the expected peak locations-as our proposed methods do.

Paradigm: Spatial Analysis
In this paradigm, the analysis is done in the spatial domain.As scaling algorithms and scaling attacks operate here, this domain naturally provides the advantage of knowing which pixels are considered by scaling algorithms and hence which are possibly modified.We identify two variants of this paradigm: A defense leverages either the adversarial modification or the clean pixels in  for detection.In the following, we examine each group in more detail.

Adversarial-Signal Driven.
The concept here is to amplify the sparse, adversarial modifications in  and then to compare the amplified image  ′ with .We have two sub-groups here.Down-and-Upscaling.We can exploit that downscaling and upscaling form antagonists in the spatial domain.That is, the scaling of  leads to an output  which corresponds to  .By upscaling  back to the original size,  ′ = upscale(downscale()), we can compare this version with .The upscaling strengthens the signal embedded by the adversary and renders it detectable through comparison with the original image.Note that the comparison function needs to account for this setup, as we describe later.
In principle, we do not need to upscale the image and could directly compare  with .In our preliminary experiments, however, we found that we can achieve better results if we upscale  to the resolution of  and conduct the comparison on the same dimensions.Moreover, some image comparison methods require the same size for the inputs.
Proposed Detection.A variety of methods exists for comparing images that would be applicable here.We find that a simple PSNR computation is particularly effective.Peak signal-to-noise ratio (PSNR) is a widely used pixel-based comparison method that returns the normalized mean-squared error between two images.Formally, it is defined as PSNR(,  ′ ) = 20 log 10 ( ) − 10 log 10 (MSE(,  ′ )). ( The constant  is the maximum of the pixel range, such as 255 for 8-bit images.The higher the PSNR value is, the more two images match.PSNR can be computed efficiently and thus provides a perfect basis for designing a detection method using down-and upscaling. Previous Approaches.Xiao et al. [24] propose two alternative comparison options.First, the intensity histograms of  and of  ′ are extracted and compared.Second, in the color-scattering method, the average distance to the image center over all pixels with the same intensity is computed.By doing this for each intensity value in  and  ′ , respectively, we obtain two vectors that can be compared.Moreover, Kim et al. [10] propose two measures that directly compare  and  ′ : the mean squared error (MSE) and the SSIM index.While the first is simply comparing  and  ′ pixel-wise, the latter aggregates information about the luminance, contrast, and structure between  and  ′ [9].
In our evaluation, the simple PSNR and SSIM outperform the other measures.Unlike suggested by prior work [10], MSE is not superior to PSNR.We assume that this conclusion is the result of a subtle implementation mistake in the evaluation metric where an integer overflow occurs.In fact, the PSNR and MSE are directly related to each other (see Equation 3).Although MSE and PSNR have the same detection performance, the PSNR score is easier to interpret.It typically lies between 0 dB and 60 dB in our evaluations.Moreover, in terms of security, a pixel-wise measure such as the PSNR is particularly robust against adaptive attacks compared to aggregation-based measures such as the histogram or colorscattering.No information are lost that could be exploited [15].
Amplifying Filter.As alternative to down-and-upscaling, Kim et al. [10] propose two methods, a minimum filter and a maximum filter, to strengthen the adversarial signal.Each filter is applied on  by iterating over the image and replacing each pixel by the minimum/maximum of its neighborhood.The outcome  ′ is compared with  using MSE or SSIM.This filtering strategy aims at amplifying the periodic modification.However, as the CSP approach, this defense overlooks the attack's root cause and misses the advantage of knowing the pixels used by scaling algorithms.A targeted filtering is possible which motivates our next group of defenses.

Clean-Signal Driven.
Another concept of the spatial paradigm is to leverage the clean pixels that a scaling attack needs to leave to keep the attack imperceptible.We can use these pixels to clean  and to compare this result with the initial version.This principle has not been studied before.It allows us to design specialized methods for the global and local attack scenario, which are presented in the following.
Proposed Global Detection.To obtain a cleansed version of , we adopt the idea of reconstructing the modified pixels using prevention filters [17].In particular, we design two methods that are based on a selective median filter and a selective random filter, respectively.Both filters repair each pixel considered by a scaling algorithm by using the neighborhood of the particular pixel.In terms of security, the reconstruction methods have the advantage of addressing the root cause of scaling attacks.They also show a strong robustness against adaptive attacks, since adversaries have to modify the neighborhood as well to bypass both filters, making the attack clearly visible [17].This security robustness also transfers to our detection defense.
For detection, we propose the following simple clean filter approach: Equipped with a prevention filter V, we can get a cleaned version  ′ = V () which is compared with the initial input .We evaluate the PSNR and the SSIM measure to compare  and  ′ .
Note that we also tested to compare  and  ′ where  ′ is the downscaled version of  ′ .Hence,  shows the new adversarial content, while  ′ shows the original content.The detection results correspond to the clean-filter method and are thus omitted.

Proposed Local Detection.
A direct application of the previous approach to the local attack scenario is not promising.As  and  ′ are here similar for the majority of pixels, a global comparison cannot sufficiently capture the difference.Instead, we propose to divide the images into patches and compare each patch individually.We propose two variants.
Patch-Clean Filter.We create  and  ′ and divide them into  patches, respectively.We compute the PSNR between each pair of patch,   = PSNR(  ,  ′  ) ∀ = 1, . . ., .The final detection score is given as |mean({  })−min({  })|.Appendix C presents the method in more detail.Note that we work on downscaled images here.We find that the smaller image size reduces the number of possible regions which improves the detection rate.
Targeted Patch-Clean Filter.As alternative, we analyze the unscaled images.However, even with patches, the scaling pixels-and thus modified pixels-are in the minority, making it difficult to detect a difference with and without scaling attack.Hence, we propose a more targeted variant and only examine the scaling pixels in each patch.Let  denote the number of patches,   &  ′  the respective patches, and Ψ the scaling-pixel selection, we compute We get the -th quantile of each vector   , denoted as q(  ), and calculate |max({q(  )}) − mean({q(  )})| as detection score.This comparison allows us to identify an unusual difference in a local area.Note that the choice of  is a hyperparameter of the method.
Remark.We also tested the down-and-upscaling concept with patches.However,  ′ = upscale(downscale()) contains a stronger noise signal due to down-and-upscaling.While this is tolerable in the global scenario, it distorts the comparison with small patches.

EVALUATION
We proceed with an empirical evaluation of the different detection methods.To obtain a comprehensive view, we study both nonadaptive and adaptive adversaries.In this section, we consider the non-adaptive case with regular attacks.Then, in Section 5, we investigate the best approaches against an adaptive adversary.

Evaluation Setup
Our evaluation follows the common design of experiments on image-scaling attacks [17,24].We evaluate the detection of attacks against popular scaling algorithms that are vulnerable to scaling attacks [17].In particular, we consider nearest-neighbor, bilinear, and bicubic scaling from the libraries OpenCV and tf.image (Ten-sorFlow), and nearest-neighbor scaling from the library Pillow.We omit Lanczos scaling which is comparable to bicubic scaling [17].
For the attacks, we adopt the evaluation setup by Quiring et al. [17], where the source and target for an attack are randomly drawn from a collection of images.As dataset for this sampling, we use photos from ImageNet [18].Compared to other datasets from computer vision, like CIFAR or CelebA, ImageNet contains significantly larger images, which is a key requirement for constructing successful scaling attacks in practice.Moreover, the dataset is very diverse, covering various image sizes and contents, such as faces, animals, persons, objects, and landscapes.
As learning model, we use a pre-trained VGG19 model [22] which is a standard benchmark in computer vision.The target size for scaling is thus 224 × 224 × 3 pixels.Note that we just use this one architecture in our experiments, since scaling attacks do not depend on the learning model's architecture.They change the input to the model.Only the input size of the model is relevant, so that we make sure to use varying scaling ratios as described later.Finally, we consider a global and local modification scenario.Both require a slightly different setup that we present in the following.
Global Modification.To obtain attack images in this scenario, we randomly sample images from ImageNet and create 1,000 sourcetarget image pairs.We ensure that the pairs have varying scaling ratios to avoid artifacts that may arise from a fixed ratio.We check that each target is unrelated to its source image by requiring different classes and predictions for each pair.After conducting the attacks, we keep only those images that are successful regarding both attack objectives, i.e., scale() ∼  and  ∼ .To this end, we use the same methodology as Quiring et al. [17].The number of successful images varies across each combination of scaling algorithm and library, so that we choose the highest possible number of images across all setups.This leads to 585 attack images for each combination of scaling algorithm and library.As unmodified reference set, we additionally select 585 further images from ImageNet.They are used to evaluate the detection of benign inputs.
Local Modification.Here, we implement the BadNets backdoor attack [7] as a representative form for local triggers that are also used in recent works [e.g., 19].We use the same 585 source images as in the previous scenario.Yet, we scale each source image to a size of 224 × 224 × 3 pixels and add a small, bounded backdoor pattern to create its respective target image (see Figure 9 in the Appendix).Finally, we conduct scaling attacks on these image pairs.As reference set, we use the same 585 benign images as before.We also verify that the so-created backdoors are effective (see Appendix D).
Evaluation Measures.To evaluate the detection performance of the considered detection methods, we equally split each attack and reference dataset into a training and test partition, respectively.The training set is used to calibrate a threshold for each detection method so that the false positive rate is 1%.This threshold is then used to evaluate the detection performance on the test set.For some detection methods, we need to decide on specific hyperparameters, such as the window width in the peak-spectrum analysis.For this calibration, we split the training set further and create a validation set.This set is used to find the optimal parameters in a grid search.We instantiate the detection methods with the best parameters (see Appendix E) and report their performance on the test set.

Global Modification Scenario
We start with the detection when source and target image are arbitrary.This is the most severe setting as an adversary can freely choose the target class of the prediction.
Results.Table 2 shows the performance for all detection methods, sorted in descending order with respect to the average accuracy.Both paradigms allow detecting scaling attacks if the entire scaled image is modified.A perfect detection is possible with our proposed peak-distance frequency analysis.However, not all methods are effective, such as the previously proposed CSP approach with an accuracy of 50%.
Analysis.A closer look on this experiment provides important insights.First, our proposed frequency methods-exploiting the known peak locations-outperform all other detection methods.Their accuracy is 100% and 99.90%.On the other hand, the CSP  approach [10] that also analyzes the frequency spectrum is comparable to random guessing.Second, the choice of the comparison function in the spatial paradigm is important.SSIM is preferable for the clean, minimum, and maximum filter.PSNR provides a higher detection accuracy with the down-and-upscaling approach.There is no difference between MSE and PSNR in our experiments.Third, patch-based defenses designed for the local scenario are partly applicable in the global case.The targeted patch-clean filter has a detection rate of 96.54%.
Ensemble of Detection Methods.Next, we combine multiple methods as ensemble to increase the diversity of detection patterns.We test three ensembles: We use the best method from each paradigm, and we use the  = {3, 4} best methods (irrespective of the paradigm).Note that we choose the methods based on the results on the training dataset and not Table 2.
The first ensemble consists of peak distance and the clean filter (with Median, SSIM).The ensemble with  = 3 consists of the first three entries in Table 2.The ensemble with  = 4 uses Down & Up with PSNR as 4th method in addition.We use two voting strategies: We report an attack if the majority of methods or if at least one method flags an input, that is, one winner takes all.
Table 3 shows the performance.The K-best ensemble with  = 4 and majority voting achieves an accuracy of 100% and a falsepositive rate of 0%.In terms of security, an ensemble thus allows a perfect detection rate while increasing the difficulty for an adaptive attack, as different paradigms and methods have to be circumvented.
Overlay Scenario.In addition, we study the variation where a scaling attack is used to embed only a low-opacity version of  into the downscaled output.More specifically, the novel target image is given as  ′ =  •  + (1 − ) • scale().The parameter  denotes the blending factor.In our experiments, we set  = 0.3 to test a challenging case with only a very small embedding of  .Table 4 shows the performance.Our frequency approaches still provide a detection rate close to 100%.Even with a very low blending factor, periodic peaks are inevitably created in the frequency spectrum.On the contrary, the spatial paradigm is more affected by the overlay scenario.A pixel-based comparison is more difficult, as  is embedded more weakly.
Comparing Scaling Algorithms and Libraries.We have presented aggregated results over all scaling algorithms and libraries so far.In Appendix F, we analyze the individual detection per scaling algorithm and library.We find no significant difference between the libraries.Moreover, the frequency methods and the leading methods in the spatial paradigm do not depend on the scaling algorithm.
Summary.The effective detection of image-scaling attacks is possible with both paradigms.The frequency paradigm allows for a perfect detection rate without any false positives in our experiments.Even in the overlay scenario, it enables spotting all attacks with high accuracy.We conclude that scaling attacks can be efficiently detected if an adversary performs a global modification.

Local Modification Scenario
In our next experiment, we test the detection performance in the challenging scenario where an adversary applies an image-scaling attack only to a small area of an image.
Results.Table 5 shows the results for all detection methods.Only our proposed frequency methods can effectively detect local scaling attacks with an average accuracy of 80.98% and 89.81%.Our patchbased approaches achieve an acceptable detection rate of 76.38% and 75.72%.The other methods are close to random guessing.
Analysis.Most methods of the spatial paradigm are not effective anymore.The reason is that scaling attacks change only a small area.Thus, most of the compared areas still correspond to the original image, making a comparison difficult.On the contrary, the frequency  approach is still effective, because even a limited, modified image area causes periodic frequency peaks.However, the impact on the frequency spectrum is weaker, so that the performance decreases compared to attacks modifying the full image.
Ensemble of Detection Methods.We also study ensembles of detection methods with local modifications.For the best-per-paradigm ensemble, we consider peak spectrum and the targeted patch-clean filter.The -best ensembles consist of the first  entries in Table 5.
We consider  = {3, 4}, as we have only four effective approaches (see Table 5).Table 6 shows the performance.With majority voting and -best ensembles, the false-positive rate can be reduced to 0% by sacrificing some accuracy.In turn, the combination of majority voting and best-per-paradigm slightly improves the accuracy, but with more false positives.Note, however, that the peak-spectrum method alone would achieve an average accuracy of 91.76% at a comparable falsepositive rate of 1.85%.Taken together, no ensemble outperforms the individual methods in any aspect.In terms of performance, the ensemble seems not directly beneficial.Yet, it provides benefits against adaptive attacks as we will see in Section 5.
Comparing Scaling Algorithms and Libraries.In Appendix F, we analyze the individual detection performance.The library has no effect, but we observe a duality with more advanced scaling algorithms: Frequency methods become better while clean-signal methods become worse.With local modifications, a defender should therefore choose the detection method based on the scaling algorithm.
Varying Backdoors.Next, we study the detection performance with more backdoors that differ in type and location.In addition to our previously used backdoor (a black box in the lower left corner), we examine (i) a black circle embedded in the upper right corner, and (ii) a rainbow-like box [13] embedded in the lower left corner.These backdoors allow us to study the impact of shape, filling, and location.The box and rainbow patterns, for instance, are common patterns in backdoor attacks [13,15,19].The first two columns in Figure 9 in the Appendix show examples for all backdoor types.Table 7 shows the performance.While the box and circle are well detectable, the rainbow-backdoor is only detected in 3 of 4 cases.We attribute this to the mixed filling, which causes weaker peaks in the frequency spectrum.Moreover, for three methods, the circle backdoor is slightly better detectable than the box.This is because the circle is larger, consuming 305 pixels compared to 225 pixels by the box.We verified this by reducing the circle size.The detection rates then become similar to the box backdoor.In summary, we study multiple backdoor types with scaling attacks.Our approaches detect them, yet the performance relies on the backdoor type.
Varying Backdoors At Train-Test Time.So far, the training dataset for calibrating the detection methods have had the same backdoor as the test dataset.In practice, this might not be realistic.As a remedy, we study an additional scenario where the training dataset is calibrated on a different backdoor than the one used in the test dataset.We study the same three backdoor types as before.
Table 8 shows the detection rate of the peak-spectrum method as a matrix for all possible backdoor combinations during train and test time.Using different backdoors has no impact on the detection.The values in each column in Table 8 are almost identical, irrespective of the backdoor at training time.The other detection methods have the same behavior and are reported in Table 14 in Appendix G.We further describe the reasons for these results in Appendix G.
Overall, we conclude that we can also detect scaling attacks if the backdoor-used to calibrate the detection-is different to the finally used backdoor from the adversary.Only the backdoor type itself, such as a circle or rainbow-like backdoor, affects the detection.
Summary.Even in the challenging scenario where only a local image area is manipulated, a reliable detection is possible.However, only four approaches are effective: our frequency approaches based on peak spectrum and peak distance, as well as both patch-clean filters.Finally, our results show that the defender does not need to have an exact knowledge of the employed backdoor.

ADAPTIVE ATTACKS
Finally, we study an adaptive attacker who is aware of the deployed detection method and adjusts the attack strategy accordingly.We examine the global and the local modification scenario again, but limit our analysis to only the successful methods from Section 4.
Note that we now have to analyze the detection rate and both goals of a scaling attack.Let Ã be the adaptive version of , an attack has to fulfill: (O1) scale( Ã) ∼  , and (O2) Ã ∼ .The second goal O2 is evaluated with the PSNR.The first goal O1 is tested by computing the attack success rate (ASR).In the global scenario, we define the ASR as the ratio of matches in the top-5 predictions from VGG19 between Ã and .In the local scenario with backdoors, we define the ASR as the ratio of successful matches of the backdoor's target class in the top-5 predictions.Appendix D provides more information on the finetuning setup to measure the ASR with backdoors.

Attacking the Frequency Paradigm
To mislead a peak analysis, an adaptive attacker has to hide the periodic traces caused by a scaling attack.In the following, we analyze different methods to achieve this.
Suppressing Frequency.We begin with a targeted attack against our peak-spectrum analysis.We shortly introduce the concept before presenting the empirical results.
Approach.The idea is to suppress the frequency spectrum in the window that is used by the defense.In particular, let  denote the frequency spectrum of the attack image and  ∈  each window used by the spectrum analysis.The spectrum is then manipulated as follows: where   is a parameter to control the reduction and [•] selects a subset of frequencies.Figure 6 illustrates the adaptive attack by setting all frequencies in the window  to zero.

Suppress frequencies in window around expected peak
Figure 6: Adaptive attack by suppressing frequencies with   = 0.0.Note that such a strong reduction is often not needed.Results.This adaptive attack has a strong impact on the detection performance, as Table 9 shows.The average detection accuracy notably decreases with a smaller value of   .Note that using   = 0.0 leads to a too strong frequency reduction, so that the detection threshold could be simply inverted, increasing the accuracy, for instance, to 66.63% (100% -33.37%) in the global case.To provide more intuition on the resulting image quality, Figure 10 in the Appendix shows an exemplary image from our evaluation for varying values of   .
Regarding the goals O1 and O2, the adaptive attack has only an impact in the global scenario.Here, the ASR decreases and the attack images loose brightness and contrast.The output images become a mixture between the original and novel content.On the contrary, in the local scenario,   has no significant impact on the ASR and the visual quality.In summary, an adaptive attacker can notably decrease the detection performance.In the global scenario, however, the attacker has to sacrifice the goals O1 and O2 to some extent, which limits the impact of the attack.
Add Frequency Peak.We continue with an attack against the peak-distance analysis.Again, we shortly introduce the concept before presenting results.
Approach.The idea is to insert an additional peak in each excerpt so that the distance between the expected and maximum peak increases.Let  ∈  be the frequency location for adding a peak in each excerpt, the spectrum is modified as follows: where   controls the strength of the added peak.For simplicity, the peak locations  ∈  are the corners of each excerpt.Figure 7 illustrates the adaptive attack.This frequency modification is hard to perceive in the whole frequency spectrum, as only very nuanced changes are performed.
Results.Table 10 shows the results of the adaptive attack.The peak addition has a strong impact on the detection performance.Already   = 75 notably decreases the performance in the global and the local modification scenario.At the same time, the peak addition has only a minor effect on the goal O1.To better explain the effect on O2, Figure 11 in the Appendix shows an example from our evaluation.As   decreases and the peaks become stronger, the images loose brightness and contrast.Still, the image content Add frequency peaks   remains intact-even at a high modification factor with   = 25.We thus conclude that O2 is also partly achieved.All in all, we identify a suitable range of 50 ⩽   ⩽ 75 where the peak-distance approach does not detect an attack and both goals of a scaling-attack are satisfied.
JPEG Compression.As a baseline, we consider a generic and simple counterattack by examining JPEG compression.It is known to have an impact on resampling detectors in multimedia forensics [11], which analyze the frequency spectrum for peaks similar to our approach.Appendix H presents the results.Compression only affects the local scenario.However, the effect on the detection performance is smaller compared to our previous two targeted adaptive attacks.

Attacking the Spatial Paradigm
Lastly, we discuss the attack surface of the other paradigm.For strengthening approaches based on down-and upscaling, an adaptive attack should not be possible.Upscaling algorithms do not suffer from the root cause that enables scaling attacks in the downscaling case.Upscaling algorithms usually use each pixel multiple times to compute the larger image, so that an attacker cannot hide a new signal.We verified the implementation of the imaging libraries OpenCV, Pillow, and tf.image (TensorFlow) and observed this behavior in their upscaling algorithms.Thus, if the downscaling reveals another content, the upscaled version will necessarily keep this content, making an adaptive attack difficult.The clean-signal driven approaches based on prevention filters inherit the security properties of the respective filter.Prior work has demonstrated that these filters withstand adaptive attackers [17].In the global scenario, we can thus conclude that the detection is robust.In the local scenario, however, the cleaning approach only works with a patch extraction.In this case, this patch extraction can introduce a new vulnerability.For example, an attacker could distribute a backdoor over multiple patches.This requires designing new backdoor methods which is beyond the scope of this paper.

Summary
Our analysis shows that the frequency paradigm, although the strongest under a static attacker, can be circumvented by an adaptive attacker.On the contrary, the spatial paradigm withstands adaptive attacks by design.Hence, we are faced with a trade-off where the frequency and spatial paradigms complement each other in detection capabilities and robustness, respectively.

RELATED WORK
Image-scaling attacks are a novel threat to the security of machinelearning systems.As a result, there exists only a small body of related work that is discussed in the following.
Attacks.Xiao et al. [24] have initially introduced image-scaling attacks.Quiring et al. [17] perform an in-depth analysis of scaling attacks and identify their root cause.We build our defenses on this understanding.Chen et al. [4] extend the original attack by studying different norms for Equation 1. Yet, these norms do not affect the attack's working principle and thus our proposed defenses.Quiring and Rieck [15] examine the application for the poisoning and backdoor scenario.This work motivates our inclusion of the local-modification scenario.Finally, Gao et al. [6] combine adversarial examples and scaling attacks.We excluded the attack, since it operates in a different threat scenario.The attack is dependent on the learning model and requires an iterative adversarial-example process.On the contrary, we focus on general scaling attacks that are model-agnostic and just create the target as scaling output.
Defenses.To fend off scaling attacks, we can either prevent or detect an attack.In the former case, Quiring et al. [17] have extensively studied prevention defenses.In the latter case, Xiao et al. [24] and Kim et al. [10] have presented first ideas, evaluated with nonadaptive attackers.We include these approaches in our comparison, but our evaluation shows that they are often ineffective.
Adversarial Learning.Scaling attacks are preprocessing attacks [17] that represent a new type of ML attack in addition to existing attacks, such as adversarial examples and poisoning [1,14,16].

CONCLUSION
This paper is the first comprehensive study on the detection of image-scaling attacks.We examine the problem from multiple viewpoints by considering various scaling algorithms, levels of modification, and attacker models.We systematize the detection and derive novel approaches based on our improved understanding.
The frequency paradigm is the strongest detection approach in any image-modification scenario under a static attack.Under an adaptive attack, it is not robust and vulnerable to evasion.The spatial paradigm is suitable for spotting global manipulations but lacks accuracy for local changes.However, it enables a robust detection under adaptive attacks.Therefore, our results motivate that both paradigms should be used as ensemble to complement each other.
Finally, we emphasize that detection should not be seen as replacement for prevention defenses.In fact, both concepts best operate in combination.While prevention methods block attacks during training and inference, approaches for detection help identify ongoing attacks and ultimately help tracking down adversaries.As a result, by combining both concepts, we can improve the security of machine-learning systems.

A FREQUENCY ANALYSIS
In the following, we derive the computation of the peak positions that an image-scaling attack inevitably introduces.We start with an uncentered view of the frequency spectrum as well as two assumptions that will be gradually relaxed: the use of nearest scaling and an integer as scaling ratio.This enables us to define a simplified image model.It mimics the blocking artifacts that a scaling attack will add in a periodic interval (recall the root-cause analysis in Section 2.2).In particular, we denote by  ∈ {0, 1} × an image where all values are set to 1, except for those on a grid in the interval   ∈ N along the vertical direction and in the interval   ∈ N along the horizontal direction: The function  corresponds to the image model for JPEG compression by Chen and Hsu [3], except for a different periodicity between the peaks in the grid.Substituting the JPEG periodicity of 8 by the periodicity   and   , we expect to observe peaks around each  1  ′ -th and  2  ′ -th position of the frequency spectrum if an image is manipulated by an image-scaling attack.More formally, we can define the following binary function Γ ∈ {0, 1} × that shows at which frequency coefficient a peak occurs: To bring this into a centered view, we shift the coordinates and get: with The constants   and   are the index of the spectrum's middle.
Let us now relax the assumptions.First, although scaling attacks manipulate more pixels for other algorithms such as bilinear and bicubic scaling, their manipulation still operates on a grid.Hence, Equation 9 can also be applied in these cases.Next, we relax the integer assumption of the scaling ratio.In practice, the ratio can also be a rational number.A closer analysis shows that the step width   and   can alternate in this case.As a result, we observe an additional periodic signal.The frequency spectrum has additional sub-peaks.Still, Equation 9 is also applicable in this case, as the major step widths correspond to   and   .Yet, we set  1 ∈ N and  2 ∈ N in Equation 9 as follows:

B FREQUENCY PEAKS IN NATURAL IMAGES
In Figure 8, we show two unmodified, benign image examples from ImageNet where the frequency domain naturally contains frequency peaks.The CSP approach would flag both images as attack.In the first image, for example, the periodic pattern of the radiator grill causes the peaks.

C PATCH-CLEAN FILTER
Here, we provide more details on the patch-clean filter.Recall that we create two versions of .First, we downscale  directly using the vulnerable scaling algorithm, yielding .Second, we apply a prevention filter V on  and downscale  ′ = V (), yielding  ′ .Hence, the adversarial modifications are present in , while  ′ is a clean version.We only use the median filter for V in our evaluation.
The filter better preserves the visual quality, which is important when small patches are used for comparison.
We apply a Gaussian filter on  to smooth it slightly, as  ′ is slightly smoothed due to the previous application of V.For simplicity, we fix the Gaussian kernel size to (3,3) and the kernel standard deviation to 0. Finally, we divide both images into patches.Let   and  ′  denote corresponding patches from the same region in  and  ′ .Let  be the number of patches.We compute   = PSNR(  ,  ′  ) ∀ = 1, . . ., .The final detection score is given as: If a scaling attack changes only a local area, only a few patches will have an unusually small PSNR value.Compared to benign images, the minimum over all patches is then significantly smaller than the mean.The difference between mean and minimum thus reveals local attacks.To obtain patches, we iterate over the image with a sliding window and extract sub-windows as patches.The sub-window size  in each direction (total side length 2 • ) and the stride  are two parameters that we determine on a validation set in our evaluation.Note that we have tested more advanced approaches to extract patches, such as k-means based segmentation or the selective search image segmentation algorithm proposed by Chou et al. [5] in the backdoor context.While the former often misses the backdoor region, the latter creates a too exhaustive list of possible regions that increases the number of false positives.

D BACKDOOR EVALUATION SETUP
Here, we describe the different setups to check that our backdoors are effective.
Static Adversary.We evaluate two settings to ensure that the backdoors work with regular image-scaling attacks.
Training from Scratch.As testing a backdoor attack by training VGG19 from scratch is computationally expensive, we resort to an equivalent, but simpler setup with the CIFAR-10 dataset [12] and the neural network architecture from Carlini and Wagner [2].In particular, we use 40,000 CIFAR images for training and embed a box backdoor on a varying number of training samples.We use the CIFAR test set for evaluating the clean accuracy on unmodified instances, and for measuring the attack success rate after embedding a backdoor.The latter shows how often a backdoored image triggers its target class.Modifying 1% of the training samples leads to a success rate of 68% while 10% lead to a success rate of 97%.The clean accuracy is not largely affected.Next, we evaluate the attack success rate when applying scaling attacks to hide the backdoor on test samples.The difference with and without scaling attack is less than 1%.We conclude that our backdoors are effective.Scaling attacks have no considerable impact on the backdoor effectivity.
Finetuning.To check the validity on VGG19 directly, we additionally test a finetuning setup to embed a backdoor.To this end, our training set consists of 350 images from the 585 backdoored images where a scaling attack is used to hide the backdoor (see Section 4.1).In addition, we collect 2,000 novel, unmodified images from ImageNet.For the backdoor, we choose a random target class as label.For finetuning, we use Adam with a small learning rate of 1e-6.To check the performance, we collect 2,000 further ImageNet images as benign test set and the remaining 235 backdoored images as attack test set.The former set is used to get the top-5 accuracy before and after finetuning.The latter set is used to measure the attack success rate, that is, how often a backdoored image can activate the target class in the top-5 predictions.We report the average and standard deviation over 10 random target classes and over all scaling libraries & algorithms.
The attack success rate is 75.66% ± 6.96%, underlining that a backdoor in combination with scaling attacks can be effectively embedded with a very simple finetuning setup.The accuracy on the benign test set drops from 95.70% to 93.48% ± 0.39%.Overall, we can conclude that our backdoor setup-with scaling attacks to hide backdoors-is effective.
Adaptive Adversary.To measure the success rate of backdoors after applying the adaptive attacks from Section 5, we adopt the prior finetuning setup.Yet, we now use the adaptive version of each attack image that contains the backdoor.For each adaptive attack with respective parameters, we run a separate finetuning setup to embed the backdoor.
We report the attack success rate for each setup in Table 9 and Table 10 in Section 5.The accuracy drop on the benign test set is comparable to the static attack before.The accuracy after finetuning over all adaptive attacks is 93.40% ± 0.14%.Due to this very low deviation across all adaptive attacks, we omit the clean accuracy in the tables in Section 5.

F COMPARING SCALING SETUPS
We have presented aggregated results over all scaling algorithms and libraries so far.Yet, scaling attacks have to modify more pixels for scaling algorithms with larger kernels such as bilinear or bicubic scaling.In this section, we therefore analyze whether specific algorithms and libraries have an impact on the detection performance.
Global Modification.Table 11 shows the detection performance for TensorFlow per scaling algorithm.The OpenCV results are similar and omitted due to lack of space.Our proposed frequency methods work for all algorithms equally well.The performance of some clean-signal driven approaches decrease with bilinear and bicubic scaling.We attribute this to the increased number of necessary pixel reconstructions by the filter.This affects the visual quality and thus the image comparison.The PSNR and the random filter are then more affected than the SSIM and the median filter.
Local Modification.Table 12 shows the results for TensorFlow.The results for OpenCV are similar and shown in Table 13.We observe a duality with more advanced algorithms: The frequency methods become better while the clean-signal methods become worse.With larger kernels, an attack has to modify more pixels.This is advantageous for frequency methods where the peaks become more prevalent.In contrast, more pixels have to be reconstructed with the filter-based methods, making a comparison more difficult.

G BACKDOORS AND FREQUENCY SPECTRUM
Table 14 shows the detection accuracy as a matrix for all backdoor combinations during training and test time.Peak spectrum is already shown in the main section in Table 8.Different backdoors at train-test time do not affect the performance.For the frequency analysis, the reason is that the peak positions do not depend on the backdoor's location or shape in the pixel domain.The frequency peaks depend on the distance between the periodic changes.Neither are the patch-based approaches affected.The compared patches are only derived from the current input image, so that a backdoor will cause an observable difference in a patch.Thus, varying locations and shapes of backdoors at test time are here detectable, too.
Figure 9 shows examples from our evaluation if different backdoors are embedded in combination with a scaling attack.The figure also shows the respective frequency spectrum.

H ADAPTIVE ATTACKS
In this section, we present additional results for the adaptive attacks.
Table 15 and Table 16 show the results of JPEG compression as adaptive attack.The global scenario is robust, while the local scenario is more affected.As strong JPEG compression removes high frequencies, it affects the local case with weaker scaling-attack peaks more.Still, the peak-spectrum method can detect more than 70% even with strong compression.Compared to our targeted attacks in Section 5, compression is less effective.
Figure 10 and Figure 11 show examples from the evaluation to provide more insights on the visual quality.

Figure 1 :
Figure 1: Image-scaling attacks: global modification (top) and local modification (bottom).Note the small box (backdoor) on the lower left that appears.Both attacks can be detected by using a frequency or spatial analysis.

Figure 2 :
Figure 2: Principle of image-scaling attacks: The adversary finds a minimal modification Δ of  such that the modified image  = ( +Δ) still looks like , but downscales to  .

Figure 4 :
Figure 4: Frequency analysis of the example in Figure 2. Plot (a) and (b) show the frequency spectrum of the source image and attack image, respectively.Plot (c) shows the marked peaks with Equation 2.

Figure 5 :
Figure 5: Frequency-based detection approaches.Plot (a): Windows around expected peaks are extracted and their frequencies are compared in relation to the overall spectrum.Plot (b): Spectrum is divided into excerpts and the distance between expected and maximum peak for each excerpt is computed.

Figure 7 :
Figure 7: Adaptive attack by adding peaks with   = 25.Note that our evaluation shows that such a strong addition is often not necessary.

Figure 9 :
Figure 9: Evaluation examples from backdoor detection.The columns show an attack image, its downscaled version, the frequency spectrum of , and the detected peaks with our frequency method.The first row depicts a scaling attack with a box backdoor in the bottom-left corner.The row in the middle shows an attack with a circle backdoor in the upper-right corner.The last row depicts an attack with the rainbow backdoor in the bottom-left corner.The plots highlight that the frequency traces do not depend on the backdoor's shape or location.

Figure 10 :
Figure 10: Evaluation examples for the adaptive attack based on suppressing peaks.The first column shows the non-adaptive, original scaling attack, while the further columns show the adaptive modification to bypass the detection.It is visible that small values of   have a clear impact in the global scenario.

Figure 11 :
Figure 11: Evaluation examples for the adaptive attack based on adding peaks.The first column shows the non-adaptive, original scaling attack, while the further columns show the adaptive modification to bypass the detection.It is visible that adding peaks reduces the brightness, but the image content remains intact.

Table 1 :
Overview of detection methods in this paper.Clean Filter * =Proposed in this paper.{.} denotes a set of possible options for a respective method.

Table 2 :
Detection performance [%] in global scenario in terms of accuracy (average + standard dev.) and false positives (average) over all scaling algorithms and libraries.* Proposed in this paper.

Table 3 :
Ensemble in global scenario.Each cell shows the average ± standard deviation over all scaling libraries and algorithms.With two paradigms, both voting methods for "best per paradigm" have identical results, so that we omit the 2nd voting method in this case.

Table 4 :
Detection accuracy in overlay scenario.Only the effective approaches with AvgAcc > 60% are shown.

Table 5 :
Detection performance [%] in local scenario in terms of accuracy (average + standard dev.) and false positives (average) over all scaling algorithms and libraries.* Proposed in this paper.

Table 7 :
Detection performance with varying backdoors (accuracy ± standard deviation).Only the four effective detection methods are shown.

Table 8 :
Performance of peak spectrum with varying backdoors at train-test time (accuracy ± standard dev.).The rows show the used backdoor at training time, the columns the backdoor at test time.

Table 9 :
Adaptive attack against peak-spectrum approach by suppressing frequency peaks (Acc.and ASR in [%], and PSNR in [dB]).

Table 10 :
Adaptive attack against the peak-distance approach by adding frequency peaks (Acc.and ASR in [%], and PSNR in [dB]).

Table 11 :
Detection accuracy per scaling algorithm in TensorFlow in the global-modification scenario.Only approaches with AvgAcc > 80% in Table 2 are presented.

Table 12 :
Detection accuracy per scaling algorithm in TensorFlow in the local-modification scenario.Only the four effective approaches from Table 5 are presented.

Table 13 :
Detection accuracy per scaling algorithm in OpenCV in the local-modification scenario.Only the four effective approaches from Table 5 are presented.

Table 14 :
Detection performance with varying backdoors at traintest time (accuracy ± standard deviation).The rows show the used backdoor at training time, the columns the backdoor at test time.Abbreviations for detection methods in first column: PD=Peak Distance, TPF=Targeted Patch-Clean Filter, PF=Patch-Clean Filter.

Table 15 :
Adaptive attack against the peak-spectrum approach based on compression (Acc.and ASR in [%], PSNR in [dB], and  is the JPEG compression level).

Table 16 :
Adaptive attack against the peak-distance approach based on compression (Acc.and ASR in [%], PSNR in [dB], and  is the JPEG compression level).