RobustBench: a standardized adversarial robustness benchmark [NeurIPS 2021 Benchmarks and Datasets Track]
RobustBench: a standardized adversarial robustness benchmark [NeurIPS 2021 Benchmarks and Datasets Track]
Francesco Croce* (University of Tübingen), Maksym Andriushchenko* (EPFL), Vikash Sehwag* (Princeton University), Edoardo Debenedetti* (EPFL), Nicolas Flammarion (EPFL), Mung Chiang (Purdue University), Prateek Mittal (Princeton University), Matthias Hein (University of Tübingen)
Leaderboard: https://robustbench.github.io/
Paper: https://arxiv.org/abs/2010.09670
❗Note❗: if you experience problems with the automatic downloading of the models from Google Drive, install the latest version of RobustBench via pip install git+https://github.com/RobustBench/robustbench.git.
May 2022: We have extended the common corruptions leaderboard on ImageNet with 3D Common Corruptions (ImageNet-3DCC). ImageNet-3DCC evaluation is interesting since (1) it includes more realistic corruptions and (2) it can be used to assess generalization of the existing models which may have overfitted to ImageNet-C. For a quickstart, click here. Note that the entries in leaderboard are still sorted according to ImageNet-C performance.
May 2022: We fixed the preprocessing issue for ImageNet corruption evaluations: previously we used resize to 256x256 and central crop to 224x224 which wasn't necessary since the ImageNet-C images are already 224x224 (see this issue). Note that this changed the ranking between the top-1 and top-2 entries.
The goal of RobustBench is to systematically track the real progress in adversarial robustness.
There are already more than 3'000 papers
on this topic, but it is still often unclear which approaches really work and which only lead to overestimated robustness.
We start from benchmarking the Linf, L2, and common corruption robustness since these are the most studied settings in the literature.
Evaluation of the robustness to Lp perturbations in general is not straightforward and requires adaptive attacks (Tramer et al., (2020)). Thus, in order to establish a reliable standardized benchmark, we need to impose some restrictions on the defenses we consider. In particular, we accept only defenses that are (1) have in general non-zero gradients wrt the inputs, (2) have a fully deterministic forward pass (i.e. no randomness) that (3) does not have an optimization loop. Often, defenses that violate these 3 principles only make gradient-based attacks harder but do not substantially improve robustness (Carlini et al., (2019)) except those that can present concrete provable guarantees (e.g. Cohen et al., (2019)).
To prevent potential overadaptation of new defenses to AutoAttack, we also welcome external evaluations based on adaptive attacks, especially where AutoAttack flags a potential overestimation of robustness. For each model, we are interested in the best known robust accuracy and see AutoAttack and adaptive attacks as complementary to each other.
RobustBench consists of two parts:
Q: How does the RobustBench leaderboard differ from the AutoAttack leaderboard?
A: The AutoAttack leaderboard was the starting point of RobustBench. Now only the RobustBench leaderboard is actively maintained.
Q: How does the RobustBench leaderboard differ from robust-ml.org?
A: robust-ml.org focuses on adaptive evaluations, but we provide a standardized benchmark. Adaptive evaluations
have been very useful (e.g., see Tramer et al., 2020) but they are also very time-consuming and not standardized by definition. Instead, we argue that one can estimate robustness accurately mostly without adaptive attacks but for this one has to introduce some restrictions on the considered models. However, we do welcome adaptive evaluations and we are always interested in showing the best known robust accuracy.
Q: How is it related to libraries like foolbox / cleverhans / advertorch?
A: These libraries provide implementations of different attacks. Besides the standardized benchmark, RobustBench
additionally provides a repository of the most robust models. So you can start using the
robust models in one line of code (see the tutorial below ).
Q: Why is Lp-robustness still interesting?
A: There are numerous interesting applications of Lp-robustness that span
transfer learning (Salman et al. (2020), Utrera et al. (2020)),
interpretability (Tsipras et al. (2018), Kaur et al. (2019), Engstrom et al. (2019)),
security (Tramèr et al. (2018), Saadatpanah et al. (2019)),
generalization (Xie et al. (2019), Zhu et al. (2019), Bochkovskiy et al. (2020)),
robustness to unseen perturbations (Xie et al. (2019), Kang et al. (2019)),
stabilization of GAN training (Zhong et al. (2020)).
Q: What about verified adversarial robustness?
A: We mostly focus on defenses which improve empirical robustness, given the lack of clarity regarding
which approaches really improve robustness and which only make some particular attacks unsuccessful.
However, we do not restrict submissions of verifiably robust models (e.g., we have Zhang et al. (2019) in our CIFAR-10 Linf leaderboard).
For methods targeting verified robustness, we encourage the readers to check out Salman et al. (2019)
and Li et al. (2020).
Q: What if I have a better attack than the one used in this benchmark?
A: We will be happy to add a better attack or any adaptive evaluation that would complement our default standardized attacks.
The goal of our Model Zoo is to simplify the usage of robust models as much as possible. Check out our Colab notebook here RobustBench: quick start for a quick introduction. It is also summarized below .
First, install the latest version of RobustBench (recommended):
pip install git+https://github.com/RobustBench/robustbench.gitor the latest stable version of RobustBench (it is possible that automatic downloading of the models may not work):
pip install git+https://github.com/RobustBench/[email protected]Now let's try to load CIFAR-10 and some quite robust CIFAR-10 models from
Carmon2019Unlabeled that achieves 59.53% robust accuracy evaluated with AA under
eps=8/255:
from robustbench.data import load_cifar10
x_test, y_test = load_cifar10(n_examples=50)
from robustbench.utils import load_model
model = load_model(model_name='Carmon2019Unlabeled', dataset='cifar10', threat_model='Linf')Let's try to evaluate the robustness of this model. We can use any favourite library for this. For example, FoolBox implements many different attacks. We can start from a simple PGD attack:
!pip install -q foolbox
import foolbox as fb
fmodel = fb.PyTorchModel(model, bounds=(0, 1))
_, advs, success = fb.attacks.LinfPGD()(fmodel, x_test.to('cuda:0'), y_test.to('cuda:0'), epsilons=[8/255])
print('Robust accuracy: {:.1%}'.format(1 - success.float().mean()))>>> Robust accuracy: 58.0%Wonderful! Can we do better with a more accurate attack?
Let's try to evaluate its robustness with a cheap version AutoAttack from ICML 2020 with 2/4 attacks (only APGD-CE and APGD-DLR):
# autoattack is installed as a dependency of robustbench so there is not need to install it separately
from autoattack import AutoAttack
adversary = AutoAttack(model, norm='Linf', eps=8/255, version='custom', attacks_to_run=['apgd-ce', 'apgd-dlr'])
adversary.apgd.n_restarts = 1
x_adv = adversary.run_standard_evaluation(x_test, y_test)>>> initial accuracy: 92.00%
>>> apgd-ce - 1/1 - 19 out of 46 successfully perturbed
>>> robust accuracy after APGD-CE: 54.00% (total time 10.3 s)
>>> apgd-dlr - 1/1 - 1 out of 27 successfully perturbed
>>> robust accuracy after APGD-DLR: 52.00% (total time 17.0 s)
>>> max Linf perturbation: 0.03137, nan in tensor: 0, max: 1.00000, min: 0.00000
>>> robust accuracy: 52.00%Note that for our standardized evaluation of Linf-robustness we use the full version of AutoAttack which is slower but
more accurate (for that just use adversary = AutoAttack(model, norm='Linf', eps=8/255)).
What about other types of perturbations? Is Lp-robustness useful there? We can evaluate the available models on more general perturbations. For example, let's take images corrupted by fog perturbations from CIFAR-10-C with the highest level of severity (5). Are different Linf robust models perform better on them?
from robustbench.data import load_cifar10c
from robustbench.utils import clean_accuracy
corruptions = ['fog']
x_test, y_test = load_cifar10c(n_examples=1000, corruptions=corruptions, severity=5)
for model_name in ['Standard', 'Engstrom2019Robustness', 'Rice2020Overfitting',
'Carmon2019Unlabeled']:
model = load_model(model_name, dataset='cifar10', threat_model='Linf')
acc = clean_accuracy(model, x_test, y_test)
print(f'Model: {model_name}, CIFAR-10-C accuracy: {acc:.1%}')>>> Model: Standard, CIFAR-10-C accuracy: 74.4%
>>> Model: Engstrom2019Robustness, CIFAR-10-C accuracy: 38.8%
>>> Model: Rice2020Overfitting, CIFAR-10-C accuracy: 22.0%
>>> Model: Carmon2019Unlabeled, CIFAR-10-C accuracy: 31.1%As we can see, all these Linf robust models perform considerably worse than the standard model on this type of corruptions. This curious phenomenon was first noticed in Adversarial Examples Are a Natural Consequence of Test Error in Noise and explained from the frequency perspective in A Fourier Perspective on Model Robustness in Computer Vision.
However, on average adversarial training does help on CIFAR-10-C. One can check this easily by loading all types of corruptions
via load_cifar10c(n_examples=1000, severity=5), and repeating evaluation on them.
3D Common Corruptions (3DCC) is a recent benchmark by Kar et al. (CVPR 2022) using scene geometry to generate realistic corruptions. You can evaluate robustness of a stand
No open issues yet, or sync has not completed.