An Open-Source Package for Textual Adversarial Attack.
An Open-Source Package for Textual Adversarial Attack.
Documentation • Features & Uses • Usage Examples • Attack Models • Toolkit Design
OpenAttack is an open-source Python-based textual adversarial attack toolkit, which handles the whole process of textual adversarial attacking, including preprocessing text, accessing the victim model, generating adversarial examples and evaluation.
⭐️ Support for all attack types. OpenAttack supports all types of attacks including sentence-/word-/character-level perturbations and gradient-/score-/decision-based/blind attack models;
⭐️ Multilinguality. OpenAttack supports English and Chinese now. Its extensible design enables quick support for more languages;
⭐️ Parallel processing. OpenAttack provides support for multi-process running of attack models to improve attack efficiency;
⭐️ Compatibility with Hugging Face. OpenAttack is fully integrated with Transformers and Datasets libraries;
⭐️ Great extensibility. You can easily attack a customized victim model on any customized dataset or develop and evaluate a customized attack model.
✅ Providing various handy baselines for attack models;
✅ Comprehensively evaluating attack models using its thorough evaluation metrics;
✅ Assisting in quick development of new attack models with the help of its common attack components;
✅ Evaluating the robustness of a machine learning model against various adversarial attacks;
✅ Conducting adversarial training to improve robustness of a machine learning model by enriching the training data with generated adversarial examples.
pip (recommended)pip install OpenAttackgit clone https://github.com/thunlp/OpenAttack.git
cd OpenAttack
python setup.py installAfter installation, you can try running demo.py to check if OpenAttack works well:
python demo.pyOpenAttack builds in some commonly used NLP models like BERT (Devlin et al. 2018) and RoBERTa (Liu et al. 2019) that have been fine-tuned on some commonly used datasets (such as SST-2). You can effortlessly conduct adversarial attacks against these built-in victim models.
The following code snippet shows how to use PWWS, a greedy algorithm-based attack model (Ren et al., 2019), to attack BERT on the SST-2 dataset (the complete executable code is here).
…Customized Victim Model
The following code snippet shows how to use PWWS to attack a customized sentiment analysis model (a statistical model built in NLTK) on SST-2 (the complete executable code is here).
…Customized Dataset
The following code snippet shows how to use PWWS to attack an existing fine-tuned sentiment analysis model on a customized dataset (the complete executable code is here).
…Multiprocessing
OpenAttack supports convenient multiprocessing to accelerate the process of adversarial attacks. The following code snippet shows how to use multiprocessing in adversarial attacks with Genetic (Alzantot et al. 2018), a genetic algorithm-based attack model (the complete executable code is here).
import OpenAttack as oa
import datasets
def dataset_mapping(x):
return {
"x": x["sentence"],
"y": 1 if x["label"] > 0.5 else 0,
}
victim = oa.loadVictim("BERT.SST")
dataset = datasets.load_dataset("sst", split="train[:20]").map(function=dataset_mapping)
attacker = oa.attackers.GeneticAttacker()
attack_eval = oa.AttackEval(attacker, victim)
# Using multiprocessing simply by specify num_workers
attack_eval.eval(dataset, visualize=True, num_workers=4)Chinese Attack
OpenAttack now supports adversarial attacks against English and Chinese victim models. Here is an example code of conducting adversarial attacks against a Chinese review classification model using PWWS.
Customized Attack Model
OpenAttack incorporates many handy components that can be easily assembled into new attack models. Here gives an example of how to design a simple attack model that shuffles the tokens in the original sentence.
Adversarial Training
OpenAttack can easily generate adversarial examples by attacking instances in the training set, which can be added to original training data set to retrain a more robust victim model, i.e., adversarial training. Here gives an example of how to conduct adversarial training with OpenAttack.
More Examples
Attack Sentence Pair Classification Models. In addition to single sentence classification models, OpenAttack support attacks against sentence pair classification models. Here is an example code of conducting adversarial attacks against an NLI model with OpenAttack.
Customized Evaluation Metric. OpenAttack supports designing a customized adversarial attack evaluation metric. Here gives an example of how to add a customized evaluation metric and use it to evaluate adversarial attacks.
According to the level of perturbations imposed on original input, textual adversarial attack models can be categorized into sentence-level, word-level, character-level attack models.
According to the accessibility to the victim model, textual adversarial attack models can be categorized into gradient-based, score-based, decision-based and blind attack models.
TAADPapers is a paper list which summarizes almost all the papers concerning textual adversarial attack and defense. You can have a look at this list to find more attack models.
Currently OpenAttack includes 15 typical attack models against text classification models that cover all attack types.
Here is the list of currently involved attack models.
decision [pdf] [code]blind [pdf] [code&data]decision [pdf] [code]score [pdf] [code]score [pdf] [code]score [pdf] [code]score [pdf] [code]score [pdf] [code]score [pdf] [code]gradient [pdf]gradient score [pdf]gradient [pdf] [code] [website]gradient [pdf] [code]score [pdf] [code&data]score [pdf] [code]The following table illustrates the comparison of the attack models.
| Model | Accessibility | Perturbation | Main Idea |
|---|---|---|---|
| SEA | Decision | Sentence | Rule-based paraphrasing |
| SCPN | Blind | Sentence | Paraphrasing |
| GAN | Decision | Sentence | Text generation by encoder-decoder |
| TextFooler | Score | Word | Greedy word substitution |
| PWWS | Score | Word | Greedy word substitution |
| Genetic | Score | Word | Genetic algorithm-based word substitution |
| SememePSO | Score | Word | Particle Swarm Optimization-based word substitution |
| BERT-ATTACK | Score | Word | Greedy contextualized word substitution |
| BAE | Score | Word | Greedy contextualized word substitution and insertion |
| FD | Gradient | Word | Gradient-based word subst |
No open issues yet, or sync has not completed.