Adversarial Texts with Gradient Methods

It has been shown in [6] that we can apply very subtle noise to images to trick deep learning into wrong prediction with very high confident. One example in shown in Figure 1. A set of adversarial images via different attacking algorithms are generated from a random image from MNIST dataset. The upper image in Clean column is the original clean image. The upper images in the following columns are adversarial images generated by the corresponding attacking algorithm, based on the first clean image, respectively. The lower image in each column is the differece between the adversarial image and the clean image, illustrated in heatmap. Below each column is the label predicted by the target model, along with probability in parenthesis. The algorithms demonstrated are FGSM [1], FGVM [2], JSMA [4] and DeepFool [3]. These are all gradient attacking methods.

Our method is straightforward. We find the adversarial text in the embedding space, e.g. [5]. Part of the difficulty of generating adversarial texts is the discreteness of the input space. Instead of working in the raw input space, we first use the above method to find candidate adversarial embeddings in the embedding space, and use nearest neighbor to find the corresponding adversarial tokens. Since the embedding space is continuous, finding adversarials is the same as images. Similar to images, most of the time we only need to modify a couple tokens (out of hundreds) to change to fool different text models.

The downside of this method is that we do not have control over the quality (e.g., syntax, semantics) of the generated sentences.

Due to the size of the dataset, we only show 100 samples for each parameter setting on the website. The rest samples could be downloaded. Note that some tokens in the clean dataset are different from the original piece of text, since these texts are also reconstructed by approximate nearest neighbor search for convenience. These does not affect the embedded vectors.