TextAttack Documentation

Get Started

• Basic-Introduction
  • What is TextAttack?
  • Where should I start?
  • NLP Attacks
  • Data Augmentation
  • Features
• Installation
  • Install with pip
  • Install from Source
  • Optional Dependencies
  • FAQ on installation
• Command-Line Usage
  • Data Augmentation with textattack augment
  • Adversarial Attacks with textattack attack
  • Training Models with textattack train
    • Available Models
      • TextAttack Models
      • transformers Models
  • Evaluating Models with textattack eval-model
  • Other Commands
    • Checkpoints and textattack attack-resume
    • Listing features with textattack list
    • Examining datasets with textattack peek-dataset
• Quick API Usage
  • Attacking a BERT model
• FAQ
  • Via Slack: Where to Ask Questions:
  • Via CLI: --help
  • Via our papers: More details on results
  • Via readthedocs: More details on APIs
  • More Concrete Questions:
    • 0. For many of the dependent library issues, the following command is the first you could try:
    • 1. How to Train
    • 2. Use Custom Models
      • Model from a file
    • 3. Use Custom Datasets
      • From a file
      • Dataset loading via other mechanism, see: more details at here
      • Custom Dataset via AttackedText class
    • 4. Benchmarking Attacks
    • 5. Create Custom or New Attacks
    • 6. The attacking is too slow

Recipes

• Attack Recipes CommandLine Use
  • Help: textattack --help
  • Running Attacks: textattack attack --help
  • Attacks and Papers Implemented (”Attack Recipes”): textattack attack --recipe [recipe_name]
  • Recipe Usage Examples
• Attack Recipes API
  • Attacks on classification models
    • A2T (A2T: Attack for Adversarial Training Recipe)
    • Alzantot Genetic Algorithm
    • Faster Alzantot Genetic Algorithm
    • BAE (BAE: BERT-Based Adversarial Examples)
    • BERT-Attack:
    • CheckList:
    • DeepWordBug
    • HotFlip
    • Improved Genetic Algorithm
    • Input Reduction
    • Kuleshov2017
    • Particle Swarm Optimization
    • PWWS
    • TextFooler (Is BERT Really Robust?)
    • TextBugger
    • CLARE Recipe
    • Pruthi2019: Combating with Robust Word Recognition
  • Attacks on sequence-to-sequence models
    • MORPHEUS2020
    • Seq2Sick
  • General
    • Imperceptible Perturbations Algorithm
• Augmenter Recipes CommandLine Use
  • Augmenting Text: textattack augment
  • Augmentation Command-Line Interface
• Augmenter Recipes API
  • Augmenter Recipes:
• TextAttack Model Zoo
  • Available Models
    • TextAttack Models
    • transformers Models
  • Evaluation Results of Available Models
    • LSTM
    • wordCNN
    • albert-base-v2
    • bert-base-uncased
    • distilbert-base-cased
    • distilbert-base-uncased
    • roberta-base
    • xlnet-base-cased
  • How we have trained the TextAttack Models
  • Training details for each TextAttack Model
  • More details on TextAttack fine-tuned NLP models (details on target NLP task, input type, output type, SOTA results on paperswithcode; model card on huggingface):

Using TextAttack

• What is an adversarial attack in NLP?
  • Terminology
  • Adversarial Examples in NLP
  • Generating adversarial examples with TextAttack
  • The future of adversarial attacks in NLP
• How to Cite TextAttack
  • Main Paper: TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
    • Our Github on TextAttack: https://github.com/QData/TextAttack
  • Our Analysis paper: Reevaluating Adversarial Examples in Natural Language
    • Our Github on Reevaluation: Reevaluating-NLP-Adversarial-Examples Github
  • Our Analysis paper: Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples
    • Our Github on benchmarking: TextAttack-Search-Benchmark Github
  • Our defense paper: Title: “Towards Improving Adversarial Training of NLP Models”
    • Code is available
    • Citations:
  • Our extended use case paper: “Expanding Scope: Adapting English Adversarial Attacks to Chinese”
    • Abstract:
    • Venue:
    • Tutorial code:
    • Citations:
• Four Components of TextAttack Attacks
  • Goal Functions
  • Constraints
  • Transformations
  • Search Methods
  • On Benchmarking Attack Recipes
  • Four components in Attack Recipes we have implemented
• Benchmarking Search Algorithms for Generating NLP Adversarial Examples
  • Title: Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples
  • Our search benchmarking result Github
  • Our benchmarking results on comparing search methods used in the past attacks.
  • Benchmarking Attack Recipes
• On Quality of Generated Adversarial Examples and How to Set Attack Contraints
  • Title: Reevaluating Adversarial Examples in Natural Language
  • Our Github on Reevaluation: Reevaluating-NLP-Adversarial-Examples Github
  • Some of our evaluation results on quality of two SOTA attack recipes
  • Some of our evaluation results on how to set constraints to evaluate NLP model’s adversarial robustness
• Making Vanilla Adversarial Training of NLP Models Feasible!
  • Title: Towards Improving Adversarial Training of NLP Models
    • Video recording of this talk: https://underline.io/events/192/sessions/7928/lecture/38377-towards-improving-adversarial-training-of-nlp-models
  • Code is available
  • Citations:
  • A2T Attack Recipe
• Lessons learned in designing TextAttack
  • Presentations on TextAttack
    • 2020: Jack Morris’ summary tutorial talk on TextAttack
    • 2021: Dr. Qi’s summary tutorial talk on TextAttack
  • Challenges in Design
  • Our design tips
  • TextAttack flowchart
  • More Details in Reference
• TextAttack Extended Functions (Multilingual)
  • Textattack Supports Multiple Model Types besides huggingface models and our textattack models:
  • Multilingual Supports
  • User defined custom inputs and models
    • Custom Datasets: Dataset from a file
      • Custom Model: from a file
  • User defined Custom attack components
  • Visulizing TextAttack generated Examples;
• How can I contribute to TextAttack?
  • Slack Channel
  • Ways to contribute
  • Submitting a new issue or feature request
    • Found a bug?
    • Do you want to add your model?
    • Do you want a new feature: a component, a recipe, or something else?
  • Start contributing! (Pull Requests)
    • Checklist
    • Tests
      • This guide was heavily inspired by the awesome transformers guide to contributing

Notebook Tutorials

• Tutorial 0: TextAttack End-To-End (Train, Eval, Attack)
  • Training
  • Evaluation
  • Attack
  • Conclusion
  • Bonus
• Tutorial 1: Transformations
  • Goal function
  • Search method
  • Transformation
  • Constraints
  • A custom transformation
  • Banana word swap
  • Using our transformation
  • Creating the goal function, model, and dataset
  • Creating the attack
  • Using the attack
  • Visualizing attack results
  • Conclusion
  • Bonus: Attacking Custom Samples
• Tutorial 2: Constraints
  • Classes of constraints
  • A new constraint
  • A custom constraint
  • NLTK and Named Entity Recognition
  • NLTK NER Example
  • Caching with @functools.lru_cache
  • Putting it all together: getting a list of Named Entity Labels from a sentence
  • Creating our NamedEntityConstraint
  • Testing our constraint
  • Conclusion
• Tutorial 3: Augmentation
  • Creating an Augmenter
  • Pre-built Augmentation Recipes
  • Conclusion
• Tutorial 4: Custom Word Embeddings
  • Importing the Model
  • Creating A Custom Dataset
  • Creating An Attack
  • Attack Results With Custom Dataset
  • Creating A Custom Word Embedding
  • Attack Results With Custom Dataset and Word Embedding
• Tutorial 5: Attacking TensorFlow models
  • Run textattack on a trained tensorflow model:
    • First: Training
    • Attacking
  • Conclusion
• Tutorial 6: Attacking scikit-learn models
  • Training
  • Attacking
  • Conclusion
• Tutorial 7: Attacking AllenNLP models
• Tutorial 8: Attacking Keras models
  • This notebook runs textattack on a trained keras model:
  • Training
  • Attacking
  • Conclusion
• Tutorial 9: Attacking multilingual models
• Tutorial10: Explaining Attacking BERT model using Captum
• Tutorial11: Attacking multilingual - Chinese NLP model using Textattack

API User Guide

• Attack
  • Attack
    • Attack
      • Attack.attack()
      • Attack.cpu_()
      • Attack.cuda_()
      • Attack.filter_transformations()
      • Attack.get_indices_to_order()
      • Attack.get_transformations()
  • AttackRecipe
    • AttackRecipe
      • AttackRecipe.build()
• Attacker
  • Attacker
    • Attacker
      • Attacker.attack_dataset()
      • Attacker.from_checkpoint()
      • Attacker.update_attack_args()
  • AttackArgs
    • AttackArgs
      • AttackArgs.create_loggers_from_args()
• AttackResult
  • AttackResult
    • AttackResult
      • AttackResult.diff_color()
      • AttackResult.goal_function_result_str()
      • AttackResult.original_text()
      • AttackResult.perturbed_text()
      • AttackResult.str_lines()
  • SuccessfulAttackResult
    • SuccessfulAttackResult
  • FailedAttackResult
    • FailedAttackResult
      • FailedAttackResult.goal_function_result_str()
      • FailedAttackResult.str_lines()
  • SkippedAttackResult
    • SkippedAttackResult
      • SkippedAttackResult.goal_function_result_str()
      • SkippedAttackResult.str_lines()
  • MaximizedAttackResult
    • MaximizedAttackResult
• Trainer
  • Trainer
    • Trainer
      • Trainer.evaluate()
      • Trainer.evaluate_step()
      • Trainer.get_eval_dataloader()
      • Trainer.get_optimizer_and_scheduler()
      • Trainer.get_train_dataloader()
      • Trainer.train()
      • Trainer.training_step()
  • TrainingArgs
    • TrainingArgs
• Datasets
  • Dataset
    • Dataset
      • Dataset.__getitem__()
      • Dataset.__len__()
  • HuggingFaceDataset
    • HuggingFaceDataset
      • HuggingFaceDataset.__getitem__()
      • HuggingFaceDataset.__len__()
• GoalFunction
  • GoalFunction
    • GoalFunction
      • GoalFunction.extra_repr_keys()
      • GoalFunction.get_output()
      • GoalFunction.get_result()
      • GoalFunction.get_results()
      • GoalFunction.init_attack_example()
  • LogitSum
    • LogitSum
  • NamedEntityRecognition
    • NamedEntityRecognition
  • TargetedStrict
    • TargetedStrict
  • TargetedBonus
    • TargetedBonus
  • ClassificationGoalFunction
    • ClassificationGoalFunction
      • ClassificationGoalFunction.extra_repr_keys()
  • TargetedClassification
    • TargetedClassification
      • TargetedClassification.extra_repr_keys()
  • UntargetedClassification
    • UntargetedClassification
  • InputReduction
    • InputReduction
      • InputReduction.extra_repr_keys()
  • TextToTextGoalFunction
    • TextToTextGoalFunction
  • MinimizeBleu
    • MinimizeBleu
      • MinimizeBleu.extra_repr_keys()
  • NonOverlappingOutput
    • NonOverlappingOutput
  • MaximizeLevenshtein
    • MaximizeLevenshtein
      • MaximizeLevenshtein.extra_repr_keys()
• Constraints
  • Constraint
    • Constraint
      • Constraint.call_many()
      • Constraint.check_compatibility()
      • Constraint.extra_repr_keys()
  • PreTransformationConstraint
    • PreTransformationConstraint
      • PreTransformationConstraint.check_compatibility()
      • PreTransformationConstraint.extra_repr_keys()
• Transformations
  • Transformation
    • Transformation
  • Composite Transformation
    • CompositeTransformation
• SearchMethod
  • SearchMethod
    • SearchMethod
      • SearchMethod.check_transformation_compatibility()
      • SearchMethod.perform_search()
      • SearchMethod.is_black_box
  • BeamSearch
    • BeamSearch
      • BeamSearch.extra_repr_keys()
      • BeamSearch.perform_search()
      • BeamSearch.is_black_box
  • GreedySearch
    • GreedySearch
      • GreedySearch.extra_repr_keys()
  • GreedyWordSwapWIR
    • GreedyWordSwapWIR
      • GreedyWordSwapWIR.check_transformation_compatibility()
      • GreedyWordSwapWIR.extra_repr_keys()
      • GreedyWordSwapWIR.perform_search()
      • GreedyWordSwapWIR.is_black_box
  • AlzantotGeneticAlgorithm
    • AlzantotGeneticAlgorithm
  • ImprovedGeneticAlgorithm
    • ImprovedGeneticAlgorithm
      • ImprovedGeneticAlgorithm.extra_repr_keys()
  • ParticleSwarmOptimization
    • ParticleSwarmOptimization
      • ParticleSwarmOptimization.check_transformation_compatibility()
      • ParticleSwarmOptimization.extra_repr_keys()
      • ParticleSwarmOptimization.perform_search()
      • ParticleSwarmOptimization.is_black_box
  • DifferentialEvolution
    • DifferentialEvolution
      • DifferentialEvolution.check_transformation_compatibility()
      • DifferentialEvolution.extra_repr_keys()
      • DifferentialEvolution.perform_search()
      • DifferentialEvolution.is_black_box

Full Reference

• textattack package
  • Subpackages
    • textattack.attack_recipes package
      • Attack Recipes Package:
      • Submodules
        • A2T (A2T: Attack for Adversarial Training Recipe)
        • A2TYoo2021
          • A2TYoo2021.build()
        • Attack Recipe Class
        • AttackRecipe
          • AttackRecipe.build()
        • Imperceptible Perturbations Algorithm
        • BadCharacters2021
          • BadCharacters2021.build()
        • BAE (BAE: BERT-Based Adversarial Examples)
        • BAEGarg2019
          • BAEGarg2019.build()
        • BERT-Attack:
        • BERTAttackLi2020
          • BERTAttackLi2020.build()
        • CheckList:
        • CheckList2020
          • CheckList2020.build()
        • Attack Chinese Recipe
        • ChineseRecipe
          • ChineseRecipe.build()
        • CLARE Recipe
        • CLARE2020
          • CLARE2020.build()
        • DeepWordBug
        • DeepWordBugGao2018
          • DeepWordBugGao2018.build()
        • Faster Alzantot Genetic Algorithm
        • FasterGeneticAlgorithmJia2019
          • FasterGeneticAlgorithmJia2019.build()
        • Attack French Recipe
        • FrenchRecipe
          • FrenchRecipe.build()
        • Alzantot Genetic Algorithm
        • GeneticAlgorithmAlzantot2018
          • GeneticAlgorithmAlzantot2018.build()
        • HotFlip
        • HotFlipEbrahimi2017
          • HotFlipEbrahimi2017.build()
        • Improved Genetic Algorithm
        • IGAWang2019
          • IGAWang2019.build()
        • Input Reduction
        • InputReductionFeng2018
          • InputReductionFeng2018.build()
        • Kuleshov2017
        • Kuleshov2017
          • Kuleshov2017.build()
        • MORPHEUS2020
        • MorpheusTan2020
          • MorpheusTan2020.build()
        • Pruthi2019: Combating with Robust Word Recognition
        • Pruthi2019
          • Pruthi2019.build()
        • Particle Swarm Optimization
        • PSOZang2020
          • PSOZang2020.build()
        • PWWS
        • PWWSRen2019
          • PWWSRen2019.build()
        • Seq2Sick
        • Seq2SickCheng2018BlackBox
          • Seq2SickCheng2018BlackBox.build()
        • Attack Spanish Recipe
        • SpanishRecipe
          • SpanishRecipe.build()
        • TextBugger
        • TextBuggerLi2018
          • TextBuggerLi2018.build()
        • TextFooler (Is BERT Really Robust?)
        • TextFoolerJin2019
          • TextFoolerJin2019.build()
    • textattack.attack_results package
      • TextAttack attack_results Package
      • Submodules
        • AttackResult Class
        • AttackResult
          • AttackResult.diff_color()
          • AttackResult.goal_function_result_str()
          • AttackResult.original_text()
          • AttackResult.perturbed_text()
          • AttackResult.str_lines()
        • FailedAttackResult Class
        • FailedAttackResult
          • FailedAttackResult.goal_function_result_str()
          • FailedAttackResult.str_lines()
        • MaximizedAttackResult Class
        • MaximizedAttackResult
        • SkippedAttackResult Class
        • SkippedAttackResult
          • SkippedAttackResult.goal_function_result_str()
          • SkippedAttackResult.str_lines()
        • SuccessfulAttackResult Class
        • SuccessfulAttackResult
    • textattack.augmentation package
      • TextAttack augmentation package:
      • Submodules
        • Augmenter Class
        • AugmentationResult
          • AugmentationResult.tempResult
        • Augmenter
          • Augmenter.augment()
          • Augmenter.augment_many()
          • Augmenter.augment_text_with_ids()
        • Augmenter Recipes:
        • BackTranscriptionAugmenter
        • BackTranslationAugmenter
        • CLAREAugmenter
        • CharSwapAugmenter
        • CheckListAugmenter
        • DeletionAugmenter
        • EasyDataAugmenter
          • EasyDataAugmenter.augment()
        • EmbeddingAugmenter
        • SwapAugmenter
        • SynonymInsertionAugmenter
        • WordNetAugmenter
    • textattack.commands package
      • TextAttack commands Package
      • Submodules
        • AttackCommand class
        • AttackCommand
          • AttackCommand.register_subcommand()
          • AttackCommand.run()
        • AttackResumeCommand class
        • AttackResumeCommand
          • AttackResumeCommand.register_subcommand()
          • AttackResumeCommand.run()
        • AugmentCommand class
        • AugmentCommand
          • AugmentCommand.register_subcommand()
          • AugmentCommand.run()
        • BenchmarkRecipeCommand class
        • BenchmarkRecipeCommand
          • BenchmarkRecipeCommand.register_subcommand()
          • BenchmarkRecipeCommand.run()
        • EvalModelCommand class
        • EvalModelCommand
          • EvalModelCommand.get_preds()
          • EvalModelCommand.register_subcommand()
          • EvalModelCommand.run()
          • EvalModelCommand.test_model_on_dataset()
        • ModelEvalArgs
          • ModelEvalArgs.batch_size
          • ModelEvalArgs.num_examples
          • ModelEvalArgs.num_examples_offset
          • ModelEvalArgs.random_seed
        • ListThingsCommand class
        • ListThingsCommand
          • ListThingsCommand.register_subcommand()
          • ListThingsCommand.run()
          • ListThingsCommand.things()
        • PeekDatasetCommand class
        • PeekDatasetCommand
          • PeekDatasetCommand.register_subcommand()
          • PeekDatasetCommand.run()
        • TextAttack CLI main class
        • main()
        • TextAttackCommand
          • TextAttackCommand.register_subcommand()
          • TextAttackCommand.run()
        • TrainModelCommand class
        • TrainModelCommand
          • TrainModelCommand.register_subcommand()
          • TrainModelCommand.run()
    • textattack.constraints package
      • Constraints
      • Subpackages
        • textattack.constraints.grammaticality package
          • Grammaticality:
          • Subpackages
            • textattack.constraints.grammaticality.language_models package
          • Submodules
            • CoLA for Grammaticality
            • COLA
            • LanguageTool Grammar Checker
            • LanguageTool
            • Part of Speech Constraint
            • PartOfSpeech
        • textattack.constraints.overlap package
          • Overlap Constraints
          • Submodules
            • BLEU Constraints
            • BLEU
            • chrF Constraints
            • chrF
            • Edit Distance Constraints
            • LevenshteinEditDistance
            • Max Perturb Words Constraints
            • MaxWordsPerturbed
            • METEOR Constraints
            • METEOR
        • textattack.constraints.pre_transformation package
          • Pre-Transformation:
          • Submodules
            • Input Column Modification
            • InputColumnModification
            • Max Modification Rate
            • MaxModificationRate
            • Max Modification Rate
            • MaxNumWordsModified
            • Max Word Index Modification
            • MaxWordIndexModification
            • Min Word Lenth
            • MinWordLength
            • Repeat Modification
            • RepeatModification
            • Stopword Modification
            • StopwordModification
            • UnmodifiableIndices
            • UnmodifablePhrases
        • textattack.constraints.semantics package
          • Semantic Constraints
          • Subpackages
            • textattack.constraints.semantics.sentence_encoders package
          • Submodules
            • BERT Score
            • BERTScore
            • Word Embedding Distance
            • WordEmbeddingDistance
      • Submodules
        • TextAttack Constraint Class
        • Constraint
          • Constraint.call_many()
          • Constraint.check_compatibility()
          • Constraint.extra_repr_keys()
        • Pre-Transformation Constraint Class
        • PreTransformationConstraint
          • PreTransformationConstraint.check_compatibility()
          • PreTransformationConstraint.extra_repr_keys()
    • textattack.datasets package
      • datasets package:
      • Subpackages
        • textattack.datasets.helpers package
          • Dataset Helpers
          • Submodules
            • Ted Multi TranslationDataset Class
            • TedMultiTranslationDataset
      • Submodules
        • Dataset Class
        • Dataset
          • Dataset.filter_by_labels_()
          • Dataset.shuffle()
        • HuggingFaceDataset Class
        • HuggingFaceDataset
          • HuggingFaceDataset.filter_by_labels_()
          • HuggingFaceDataset.shuffle()
        • get_datasets_dataset_columns()
    • textattack.goal_function_results package
      • Goal Function Result package:
      • Subpackages
        • textattack.goal_function_results.custom package
          • Custom Goal Function Result package:
          • Submodules
            • LogitSumGoalFunctionResult Class
            • LogitSumGoalFunctionResult
            • NamedEntityRecognitionoalFunctionResult Class
            • NamedEntityRecognitionGoalFunctionResult
            • TargetedBonusGoalFunctionResult Class
            • TargetedBonusGoalFunctionResult
            • TargetedStrictGoalFunctionResult Class
            • TargetedStrictGoalFunctionResult
      • Submodules
        • ClassificationGoalFunctionResult Class
        • ClassificationGoalFunctionResult
          • ClassificationGoalFunctionResult.get_colored_output()
          • ClassificationGoalFunctionResult.get_text_color_input()
          • ClassificationGoalFunctionResult.get_text_color_perturbed()
        • GoalFunctionResult class
        • GoalFunctionResult
          • GoalFunctionResult.get_colored_output()
          • GoalFunctionResult.get_text_color_input()
          • GoalFunctionResult.get_text_color_perturbed()
        • GoalFunctionResultStatus
          • GoalFunctionResultStatus.MAXIMIZING
          • GoalFunctionResultStatus.SEARCHING
          • GoalFunctionResultStatus.SKIPPED
          • GoalFunctionResultStatus.SUCCEEDED
        • TextToTextGoalFunctionResult Class
        • TextToTextGoalFunctionResult
          • TextToTextGoalFunctionResult.get_colored_output()
          • TextToTextGoalFunctionResult.get_text_color_input()
          • TextToTextGoalFunctionResult.get_text_color_perturbed()
    • textattack.goal_functions package
      • Goal Functions
      • Subpackages
        • textattack.goal_functions.classification package
          • Goal fucntion for Classification
          • Submodules
            • Determine for if an attack has been successful in Classification
            • ClassificationGoalFunction
            • Determine if an attack has been successful in Hard Label Classficiation.
            • HardLabelClassification
            • Determine if maintaining the same predicted label (input reduction)
            • InputReduction
            • Determine if an attack has been successful in targeted Classification
            • TargetedClassification
            • Determine successful in untargeted Classification
            • UntargetedClassification
        • textattack.goal_functions.custom package
          • Custom goal functions
          • Submodules
            • Goal Function for Logit sum
            • LogitSum
            • Goal Function for NamedEntityRecognition
            • NamedEntityRecognition
            • Goal Function for Targeted classification with bonus score
            • TargetedBonus
            • Goal Function for Strict targeted classification
            • TargetedStrict
        • textattack.goal_functions.text package
          • Goal Function for Text to Text case
          • Submodules
            • MaximizeLevenshtein
            • Goal Function for Attempts to minimize the BLEU score
            • MinimizeBleu
            • get_bleu()
            • Goal Function for seq2sick
            • NonOverlappingOutput
            • get_words_cached()
            • word_difference_score()
            • Goal Function for TextToText
            • TextToTextGoalFunction
      • Submodules
        • GoalFunction Class
        • GoalFunction
          • GoalFunction.clear_cache()
          • GoalFunction.extra_repr_keys()
          • GoalFunction.get_output()
          • GoalFunction.get_result()
          • GoalFunction.get_results()
          • GoalFunction.init_attack_example()
    • textattack.llms package
      • Large Language Models
      • Submodules
        • ChatGptWrapper
        • HuggingFaceLLMWrapper
    • textattack.loggers package
      • Misc Loggers: Loggers track, visualize, and export attack results.
      • Submodules
        • Managing Attack Logs.
        • AttackLogManager
          • AttackLogManager.add_output_csv()
          • AttackLogManager.add_output_file()
          • AttackLogManager.add_output_summary_json()
          • AttackLogManager.disable_color()
          • AttackLogManager.enable_stdout()
          • AttackLogManager.enable_visdom()
          • AttackLogManager.enable_wandb()
          • AttackLogManager.flush()
          • AttackLogManager.log_attack_details()
          • AttackLogManager.log_result()
          • AttackLogManager.log_results()
          • AttackLogManager.log_sep()
          • AttackLogManager.log_summary()
          • AttackLogManager.log_summary_rows()
          • AttackLogManager.metrics
        • Attack Logs to CSV
        • CSVLogger
          • CSVLogger.close()
          • CSVLogger.flush()
          • CSVLogger.log_attack_result()
        • Attack Logs to file
        • FileLogger
          • FileLogger.close()
          • FileLogger.flush()
          • FileLogger.log_attack_result()
          • FileLogger.log_sep()
          • FileLogger.log_summary_rows()
        • Attack Summary Results Logs to Json
        • JsonSummaryLogger
          • JsonSummaryLogger.close()
          • JsonSummaryLogger.flush()
          • JsonSummaryLogger.log_summary_rows()
        • Attack Logger Wrapper
        • Logger
          • Logger.close()
          • Logger.flush()
          • Logger.log_attack_result()
          • Logger.log_hist()
          • Logger.log_sep()
          • Logger.log_summary_rows()
        • Attack Logs to Visdom
        • VisdomLogger
          • VisdomLogger.bar()
          • VisdomLogger.flush()
          • VisdomLogger.hist()
          • VisdomLogger.log_attack_result()
          • VisdomLogger.log_hist()
          • VisdomLogger.log_summary_rows()
          • VisdomLogger.table()
          • VisdomLogger.text()
        • port_is_open()
        • Attack Logs to WandB
        • WeightsAndBiasesLogger
          • WeightsAndBiasesLogger.log_attack_result()
          • WeightsAndBiasesLogger.log_sep()
          • WeightsAndBiasesLogger.log_summary_rows()
    • textattack.metrics package
      • metrics package: to calculate advanced metrics for evaluting attacks and augmented text
      • Subpackages
        • textattack.metrics.attack_metrics package
          • attack_metrics package:
          • Submodules
            • Metrics on AttackQueries
            • AttackQueries
            • Metrics on AttackSuccessRate
            • AttackSuccessRate
            • Metrics on perturbed words
            • WordsPerturbed
        • textattack.metrics.quality_metrics package
          • Metrics on Quality package
          • Submodules
            • BERTScoreMetric class:
            • BERTScoreMetric
            • MeteorMetric class:
            • MeteorMetric
            • Perplexity Metric:
            • Perplexity
            • USEMetric class:
            • SBERTMetric
            • USEMetric class:
            • USEMetric
      • Submodules
        • Metric Class
        • Metric
          • Metric.calculate()
        • Attack Metric Quality Recipes:
        • AdvancedAttackMetric
          • AdvancedAttackMetric.calculate()
    • textattack.models package
      • Models
        • Models User-specified
        • Models Pre-trained
        • Model Wrappers
      • Subpackages
        • textattack.models.helpers package
          • Moderl Helpers
          • Submodules
            • Glove Embedding
            • EmbeddingLayer
            • GloveEmbeddingLayer
            • LSTM 4 Classification
            • LSTMForClassification
            • T5 model trained to generate text from text
            • T5ForTextToText
            • Util function for Model Wrapper
            • load_cached_state_dict()
            • Word CNN for Classification
            • CNNTextLayer
            • WordCNNForClassification
        • textattack.models.tokenizers package
          • Tokenizers for Model Wrapper
          • Submodules
            • Glove Tokenizer
            • GloveTokenizer
            • WordLevelTokenizer
            • T5 Tokenizer
            • T5Tokenizer
        • textattack.models.wrappers package
          • Model Wrappers Package
          • Submodules
            • HuggingFace Model Wrapper
            • HuggingFaceModelWrapper
            • ModelWrapper class
            • ModelWrapper
            • PyTorch Model Wrapper
            • PyTorchModelWrapper
            • scikit-learn Model Wrapper
            • SklearnModelWrapper
            • TensorFlow Model Wrapper
            • TensorFlowModelWrapper
    • textattack.prompt_augmentation package
      • Prompt Augmentation
      • Submodules
        • PromptAugmentationPipeline
    • textattack.search_methods package
      • Search Methods
      • Submodules
        • Reimplementation of search method from Generating Natural Language Adversarial Examples
        • AlzantotGeneticAlgorithm
        • Beam Search
        • BeamSearch
          • BeamSearch.extra_repr_keys()
          • BeamSearch.perform_search()
          • BeamSearch.is_black_box
        • DifferentialEvolution
          • DifferentialEvolution.check_transformation_compatibility()
          • DifferentialEvolution.extra_repr_keys()
          • DifferentialEvolution.perform_search()
          • DifferentialEvolution.is_black_box
        • Genetic Algorithm Word Swap
        • GeneticAlgorithm
          • GeneticAlgorithm.check_transformation_compatibility()
          • GeneticAlgorithm.extra_repr_keys()
          • GeneticAlgorithm.perform_search()
          • GeneticAlgorithm.is_black_box
        • Greedy Search
        • GreedySearch
          • GreedySearch.extra_repr_keys()
        • Greedy Word Swap with Word Importance Ranking
        • GreedyWordSwapWIR
          • GreedyWordSwapWIR.check_transformation_compatibility()
          • GreedyWordSwapWIR.extra_repr_keys()
          • GreedyWordSwapWIR.perform_search()
          • GreedyWordSwapWIR.is_black_box
        • Reimplementation of search method from Xiaosen Wang, Hao Jin, Kun He (2019).
        • ImprovedGeneticAlgorithm
          • ImprovedGeneticAlgorithm.extra_repr_keys()
        • Particle Swarm Optimization
        • ParticleSwarmOptimization
          • ParticleSwarmOptimization.check_transformation_compatibility()
          • ParticleSwarmOptimization.extra_repr_keys()
          • ParticleSwarmOptimization.perform_search()
          • ParticleSwarmOptimization.is_black_box
        • normalize()
        • Population based Search abstract class
        • PopulationBasedSearch
        • PopulationMember
          • PopulationMember.num_words
          • PopulationMember.score
          • PopulationMember.words
        • Search Method Abstract Class
        • SearchMethod
          • SearchMethod.check_transformation_compatibility()
          • SearchMethod.get_victim_model()
          • SearchMethod.perform_search()
          • SearchMethod.is_black_box
    • textattack.shared package
      • Shared TextAttack Functions
      • Subpackages
        • textattack.shared.utils package
          • Submodules
            • LazyLoader
            • load_module_from_file()
            • download_from_s3()
            • download_from_url()
            • http_get()
            • path_in_cache()
            • s3_url()
            • set_cache_dir()
            • unzip_file()
            • get_textattack_model_num_labels()
            • hashable()
            • html_style_from_dict()
            • html_table_from_rows()
            • load_textattack_model_from_path()
            • set_seed()
            • sigmoid()
            • ANSI_ESCAPE_CODES
            • ReprMixin
            • TextAttackFlairTokenizer
            • add_indent()
            • check_if_punctuations()
            • check_if_subword()
            • color_from_label()
            • color_from_output()
            • color_text()
            • default_class_repr()
            • flair_tag()
            • has_letter()
            • is_one_word()
            • process_label_name()
            • strip_BPE_artifacts()
            • words_from_text()
            • zip_flair_result()
            • zip_stanza_result()
            • batch_model_predict()
      • Submodules
        • Attacked Text Class
        • AttackedText
          • AttackedText.align_with_model_tokens()
          • AttackedText.all_words_diff()
          • AttackedText.convert_from_original_idxs()
          • AttackedText.delete_word_at_index()
          • AttackedText.first_word_diff()
          • AttackedText.first_word_diff_index()
          • AttackedText.free_memory()
          • AttackedText.generate_new_attacked_text()
          • AttackedText.get_deletion_indices()
          • AttackedText.insert_text_after_word_index()
          • AttackedText.insert_text_before_word_index()
          • AttackedText.ith_word_diff()
          • AttackedText.ner_of_word_index()
          • AttackedText.pos_of_word_index()
          • AttackedText.printable_text()
          • AttackedText.replace_word_at_index()
          • AttackedText.replace_words_at_indices()
          • AttackedText.text_after_word_index()
          • AttackedText.text_until_word_index()
          • AttackedText.text_window_around_index()
          • AttackedText.words_diff_num()
          • AttackedText.words_diff_ratio()
          • AttackedText.SPLIT_TOKEN
          • AttackedText.column_labels
          • AttackedText.newly_swapped_words
          • AttackedText.num_words
          • AttackedText.text
          • AttackedText.tokenizer_input
          • AttackedText.words
          • AttackedText.words_per_input
        • Misc Checkpoints
        • AttackCheckpoint
          • AttackCheckpoint.load()
          • AttackCheckpoint.save()
          • AttackCheckpoint.dataset_offset
          • AttackCheckpoint.datetime
          • AttackCheckpoint.num_failed_attacks
          • AttackCheckpoint.num_maximized_attacks
          • AttackCheckpoint.num_remaining_attacks
          • AttackCheckpoint.num_skipped_attacks
          • AttackCheckpoint.num_successful_attacks
          • AttackCheckpoint.results_count
        • Shared data fields
        • Misc Validators
        • transformation_consists_of()
        • transformation_consists_of_word_swaps()
        • transformation_consists_of_word_swaps_and_deletions()
        • transformation_consists_of_word_swaps_differential_evolution()
        • validate_model_goal_function_compatibility()
        • validate_model_gradient_word_swap_compatibility()
        • Shared loads word embeddings and related distances
        • AbstractWordEmbedding
          • AbstractWordEmbedding.get_cos_sim()
          • AbstractWordEmbedding.get_mse_dist()
          • AbstractWordEmbedding.index2word()
          • AbstractWordEmbedding.nearest_neighbours()
          • AbstractWordEmbedding.word2index()
        • GensimWordEmbedding
          • GensimWordEmbedding.get_cos_sim()
          • GensimWordEmbedding.get_mse_dist()
          • GensimWordEmbedding.index2word()
          • GensimWordEmbedding.nearest_neighbours()
          • GensimWordEmbedding.word2index()
        • WordEmbedding
          • WordEmbedding.counterfitted_GLOVE_embedding()
          • WordEmbedding.get_cos_sim()
          • WordEmbedding.get_mse_dist()
          • WordEmbedding.index2word()
          • WordEmbedding.nearest_neighbours()
          • WordEmbedding.word2index()
          • WordEmbedding.PATH
    • textattack.transformations package
      • Transformations
      • Subpackages
        • textattack.transformations.sentence_transformations package
          • sentence_transformations package
          • Submodules
            • BackTranscription class
            • BackTranscription
            • BackTranslation class
            • BackTranslation
            • SentenceTransformation class
            • SentenceTransformation
        • textattack.transformations.word_insertions package
          • word_insertions package
          • Submodules
            • WordInsertion Class
            • WordInsertion
            • WordInsertionMaskedLM Class
            • WordInsertionMaskedLM
            • WordInsertionRandomSynonym Class
            • WordInsertionRandomSynonym
            • check_if_one_word()
        • textattack.transformations.word_merges package
          • word_merges package
          • Submodules
            • Word Merge
            • WordMerge
            • WordMergeMaskedLM class
            • WordMergeMaskedLM
            • find_merge_index()
        • textattack.transformations.word_swaps package
          • word_swaps package
          • Subpackages
            • textattack.transformations.word_swaps.chn_transformations package
          • Submodules
            • Word Swap
            • WordSwap
            • Word Swap by Changing Location
            • WordSwapChangeLocation
            • idx_to_words()
            • Word Swap by Changing Name
            • WordSwapChangeName
            • Word Swap by Changing Number
            • WordSwapChangeNumber
            • idx_to_words()
            • Word Swap by Contraction
            • WordSwapContract
            • Word Swap by Invisible Deletions
            • WordSwapDeletions
            • Word Swap for Differential Evolution
            • WordSwapDifferentialEvolution
            • Word Swap by Embedding
            • WordSwapEmbedding
            • recover_word_case()
            • Word Swap by Extension
            • WordSwapExtend
            • Word Swap by Gradient
            • WordSwapGradientBased
            • Word Swap by Homoglyph
            • WordSwapHomoglyphSwap
            • Word Swap by OpenHowNet
            • WordSwapHowNet
            • recover_word_case()
            • Word Swap by inflections
            • WordSwapInflections
            • Word Swap by Invisible Characters
            • WordSwapInvisibleCharacters
            • Word Swap by BERT-Masked LM.
            • WordSwapMaskedLM
            • recover_word_case()
            • Word Swap by Neighboring Character Swap
            • WordSwapNeighboringCharacterSwap
            • Word Swap by swaps characters with QWERTY adjacent keys
            • WordSwapQWERTY
            • Word Swap by Random Character Deletion
            • WordSwapRandomCharacterDeletion
            • Word Swap by Random Character Insertion
            • WordSwapRandomCharacterInsertion
            • Word Swap by Random Character Substitution
            • WordSwapRandomCharacterSubstitution
            • Word Swap by Invisible Reorderings
            • WordSwapReorderings
            • Word Swap by swapping synonyms in WordNet
            • WordSwapWordNet
      • Submodules
        • Composite Transformation
        • CompositeTransformation
        • Transformation Abstract Class
        • Transformation
          • Transformation.deterministic
        • word deletion Transformation
        • WordDeletion
        • Word Swap Transformation by swapping the order of words
        • WordInnerSwapRandom
          • WordInnerSwapRandom.deterministic
  • Submodules
    • Attack Class
    • Attack
      • Attack.attack()
      • Attack.clear_cache()
      • Attack.cpu_()
      • Attack.cuda_()
      • Attack.filter_transformations()
      • Attack.get_indices_to_order()
      • Attack.get_transformations()
    • AttackArgs Class
    • AttackArgs
      • AttackArgs.create_loggers_from_args()
      • AttackArgs.attack_n
      • AttackArgs.checkpoint_dir
      • AttackArgs.checkpoint_interval
      • AttackArgs.csv_coloring_style
      • AttackArgs.disable_stdout
      • AttackArgs.enable_advance_metrics
      • AttackArgs.log_summary_to_json
      • AttackArgs.log_to_csv
      • AttackArgs.log_to_txt
      • AttackArgs.log_to_visdom
      • AttackArgs.log_to_wandb
      • AttackArgs.metrics
      • AttackArgs.num_examples
      • AttackArgs.num_examples_offset
      • AttackArgs.num_successful_examples
      • AttackArgs.num_workers_per_device
      • AttackArgs.parallel
      • AttackArgs.query_budget
      • AttackArgs.random_seed
      • AttackArgs.shuffle
      • AttackArgs.silent
    • CommandLineAttackArgs
    • Attacker Class
    • Attacker
      • Attacker.attack_dataset()
      • Attacker.attack_interactive()
      • Attacker.from_checkpoint()
      • Attacker.update_attack_args()
    • attack_from_queue()
    • pytorch_multiprocessing_workaround()
    • set_env_variables()
    • AugmenterArgs Class
    • AugmenterArgs
      • AugmenterArgs.enable_advanced_metrics
      • AugmenterArgs.exclude_original
      • AugmenterArgs.fast_augment
      • AugmenterArgs.high_yield
      • AugmenterArgs.input_column
      • AugmenterArgs.input_csv
      • AugmenterArgs.interactive
      • AugmenterArgs.output_csv
      • AugmenterArgs.overwrite
      • AugmenterArgs.pct_words_to_swap
      • AugmenterArgs.random_seed
      • AugmenterArgs.recipe
      • AugmenterArgs.transformations_per_example
    • DatasetArgs Class
    • DatasetArgs
      • DatasetArgs.dataset_by_model
      • DatasetArgs.dataset_from_file
      • DatasetArgs.dataset_from_huggingface
      • DatasetArgs.dataset_split
      • DatasetArgs.filter_by_labels
    • ModelArgs Class
    • ModelArgs
      • ModelArgs.model
      • ModelArgs.model_from_file
      • ModelArgs.model_from_huggingface
    • Trainer Class
    • Trainer
      • Trainer.evaluate()
      • Trainer.evaluate_step()
      • Trainer.get_eval_dataloader()
      • Trainer.get_optimizer_and_scheduler()
      • Trainer.get_train_dataloader()
      • Trainer.train()
      • Trainer.training_step()
    • TrainingArgs Class
    • CommandLineTrainingArgs
      • CommandLineTrainingArgs.output_dir
    • TrainingArgs
      • TrainingArgs.alpha
      • TrainingArgs.attack_epoch_interval
      • TrainingArgs.attack_num_workers_per_device
      • TrainingArgs.checkpoint_interval_epochs
      • TrainingArgs.checkpoint_interval_steps
      • TrainingArgs.early_stopping_epochs
      • TrainingArgs.gradient_accumulation_steps
      • TrainingArgs.learning_rate
      • TrainingArgs.load_best_model_at_end
      • TrainingArgs.log_to_tb
      • TrainingArgs.log_to_wandb
      • TrainingArgs.logging_interval_step
      • TrainingArgs.num_clean_epochs
      • TrainingArgs.num_epochs
      • TrainingArgs.num_train_adv_examples
      • TrainingArgs.num_warmup_steps
      • TrainingArgs.output_dir
      • TrainingArgs.parallel
      • TrainingArgs.per_device_eval_batch_size
      • TrainingArgs.per_device_train_batch_size
      • TrainingArgs.query_budget_train
      • TrainingArgs.random_seed
      • TrainingArgs.save_last
      • TrainingArgs.tb_log_dir
      • TrainingArgs.wandb_project
      • TrainingArgs.weight_decay
    • default_output_dir()