CS 229: MilestoneLearning Adversarially Robust and Rich Image Transformations for Object

Classification

Matthew Tan (mratan), Kimberly Te (kimte), and Nicholas Lai (nicklai)Category: Computer Vision

December 14, 2019

Abstract

Adversarial attacks pose major safety and ethical dan-gers in object classification-based systems, such as fa-cial recognition and autonomous driving. Our projectaims to learn adversarially robust image transforma-tions as a means of defense against these attacks forthe image classification tasks. We analyzed the use oflossy compression techniques to “clean” adversarialimages prior to feeding them to the object classificationsystems. Specifically, we implemented and evaluatedblack-box compression techniques, namely JPEG com-pression, Gaussian smoothing, K-Means, total vari-ance minimization (TVM), and Vector-Quantized Vari-ational Autoencoders (VQ-VAEs). These defenses weretested against various white-box adversarial attacks(FGSM, PGD, CarliniWagnerL2Attack, and Deep-Fool). on the MNIST and CIFAR-10 datasets. We usedindustry standard models for MNIST and CIFAR-10as our baseline models. Our results showed that our de-fenses were capable of cleaning adversarial noise fromthe images to improve accuracy, suggesting potentialagainst adversarial attacks. TVM and JPEG had thehighest accuracies, where TVM was able to achieve atleast 70% accuracy on both MNIST and CIFAR-10 de-spite the presence of strong adversarial attacks.

IntroductionWith the increasing adoption of machine learning anddeep learning systems in safety critical applications, [15],incentives to abuse these systems have also increased,where adversarial attacks can cause these systems to mis-behave. This poses ethical concerns and safety risks ontheir applications in real-world systems, such as health-care, sensors, autonomous driving, and facial recognition[5]. Despite major advancements of deep-learning in ob-ject classification, state-of-the-art algorithms are incred-ibly susceptible to adversarial perturbations [2] [7] [20].These adversarial perturbations cause algorithms to out-put highly confident erroneous predictions and underminethe effectiveness of neural network models (See Figure1). Therefore, there is a need to build defenses againstadversarial attacks and to develop more adversarially ro-bust models. Understanding how to defend against theseadversarial attacks is paramount to building safe, ethicaland widespread systems.

Therefore, the purpose of our project was to investi-gate possible defenses towards developing adversariallyrobust systems for object classification 1. Specifically, we

1By adversarially robust, we mean that the model can clas-

Figure 1: CIFAR-10 example of FGSM attack and result-ing incorrect prediction. Attacks can seem inperceptibleto the human eye.

focused on applying input image transformations as de-fenses to ”clean” the adversarial perturbations or noiseusing lossy compression and neural-network based tech-niques. Notably, we investigated the Vector QuantizedVariational Autoencoders as a potential novel defense foradversarial attacks. Finally, we then examined these ad-versarial image transformation methods against state-of-the-art adversarial attacks on the models.

Related WorkAdversarial examples are defined as inputs specificallydesigned by an adversary to make a model predict erro-neously [1]. For this project, we follow the formal defini-tion, where adversarial examples are inputs x with smallperturbations η such that a prediction on x + η outputshigh probabilities for a class is different from the classpredicted on x [6].

These adversarial attacks can be generally classifiedinto two threat models: black-box attacks and white-boxattacks. In the black-box threat model, the attacker doesnot have access to the model when designing the inputs.On the other hand, in the white box model, the attackerhas full access to the model and can exploit this informa-tion in customizing their inputs. For this project work, wefocus on the white-box threat model due to risk transfer-ability to black-box threat models. In particular, [4] hasshown that adversarial examples transfer between mod-els. Thus an adversarial example that works on one modelmay work for another different model. Because of thenon-differentiability of some of defense models, for fair-

sify correctly in the presence of adversarial perturbations ofl∞ < η for some small η

1

ness, we treat all defenses as white-box and do not pro-vide it to the attacking model.

While recent works have shown success in creatingrobust classifiers and defenses on simple datasets [22][6], currently, there is no defense that is both generaliz-able and effective against all adversarial examples. Re-cent work has developed a variety of techniques to mit-igate the impact of these attacks. Broadly, we can clas-sify these techniques into: obfuscating gradients [2], man-ifold projection using GANs / autoencoders [10] [9], andother methods (K-Nearest Neighbors, convex optimiza-tion) [19] [18] [23]. However, recent work has shownneural network models that rely on obfuscating gradientscan be circumvented by approximating the gradients [2].Moreover, several of these other works [6] [23] proposemethods with theoretical guarantees against certain typesof attacks, however none have been shown to scale effec-tively to larger datasets.

Our use JPEG compression, traditional image transfor-mations, and quantized encoders as adversarial defense.

Relevant to our work, Shin et al. [17] shows that JPEGcompression is a simple yet powerful defense on the Ima-genet dataset against Fast Gradient Sign Method (FGSM)[7] and Iterative FGSM (I-FGSm) [11]. Our work extendsthis by further exploring JPEG compression against newstate-of-the-art attack methods such as Projected GradientDescent (PGD) [13] and Deep Fool Attack [14].

Guo et al. found that traditional transformations to in-put images could act as potential adversarial defenses,such as cropping, image quilting and total variance mini-mization (TVM) [8]. Our work further explores the TVMapproach on additional datasets and new attack methods(PGD), and utilizes other image transformations, such asK-Means and a Gaussian blur filter.

Vector Quantized-Variational AutoEncoders (VQ-VAE) [21] is a variant of Variational Autoencoders(VAE). It learns two models, an encoder and a decoderthat uses a discrete latent space. Previous work has shownthe potential of VAE for adversarial defense [12] withlimited results. We hypothesize that the quantized natureof VQ-VAEs can serve as a ”lossy” compression mecha-nism similar to JPEG. From our current knowledge, ourwork is a novel potential approach at using VQ-VAE foradversarial defense.

Dataset and FeaturesWe utilized two datasets, MNIST and CIFAR-10, due totheir versatile application for object classification (SeeFigure 2). MNIST is a dataset of 60,000 greyscale hand-written digits from one to nine, each 1 × 28 × 28 pixels,split into 50,000 training examples and 10,000 test exam-ples. CIFAR-10 is a dataset of 60,000 RGB images, each3×32×32 pixels, with ten possible classes of object. Thedataset is divided into 50,000 training and 10,000 test im-ages for each class.

To minimize the computational cost of evaluating theeffectiveness of our defensive transformations, we testedon a subset of 1,000 randomly sampled examples. 2

2We attempted to running the full dataset for one model, but

Figure 2: Examples of VQ-VAE reconstruction onMNIST (first row), CIFAR-10 (second row) without anyadversarial perturbations.

.

Since we used convolutional neural networks as the ba-sis for our image models, we extract features from thedata itself rather than imposing features as would be donein supervised learning. Furthermore, in many of the stud-ied attack methods, they rely on these same extracted fea-tures (or gradients) to adversarially transform images. Assuch, the features that we used for each run vary depend-ing on the dataset and initialization parameters we use.

MethodsModelsFor our classifier models, we used two models for thetwo datasets. First, for the MNIST dataset, we trained a2 Conv + 2 FC network for 10 iterations with a batch sizeof 64 and learning rate of 0.01. We also used an SGD op-timizer with a momentum of 0.5. The images are also nor-malized before being fed into the model. 3 This achievedan accuracy of 99% on the test set. This served as ourbaseline accuracy for the MNIST dataset.

For the CIFAR dataset, we utilized a pre-trainedDenseNet based model and ran it for 150 iterations witha learning rate of 0.1 and batch size of 128. We then usea SGD optimizer with momentum 0.9 and weight decay5e-4. Our implementation also incorporates various dataaugmentation methods such as random cropping, randomflipping. Finally the images are normalized before be-ing fed into the model. 4 This achieved a baseline ac-curacy of 85% in the CIFAR-10 dataset. The accuracywe achieve are consistent with state-of-the-art models forthese datasets.

Defense PipelineOur defense pipelline consists of running our baselinemodel on the original images, adversarial images, andtransformed images. Our methodology was implemented

it yielded similar results and this took more than 24 hours, wherecomputational resources were limited.

3We use the implementation from Pytorch exampleshttps://github.com/pytorch/examples/tree/master/mnist

4We use implementation from https://github.com/kuangliu/pytorch-cifar

2

in Pytorch 5 6. A flowchart of our work process imple-mentation can be seen in Figure 3.

Figure 3: Our project model pipeline process. After trans-forming original image input into adversarial examples,we utilized our five defenses to ”clean” the adversarial at-tributes to try to restore the original accuracy. As givenabove, we produced accuracies for the original images,adversarial images, and defended images.

We examined five different lossy transformation tech-niques, including a neural-network based VQ-VAE:

1. JPEG Compression: is a standard lossy image com-pression technique based on discrete cosine transform.The quality of the compression is controlled as a hyper-parameter with 100% being lossless compression. Weuse JPEG compression to transform our adversarial im-ages into a cleaned variant.

2. Gaussian Smoothing: widely used technique in com-puter graphics for image smoothing. It acts as a low-pass filter that can smooth out high frequency noises.The quality of this transformation is controlled by thethe parameters of the Gaussian filter used (Σ).

3. K-Means compression: lossy image compression al-gorithm that uses K-Means to clusters colors, quantizethe colors, and compress the image. The number ofpossible clusters (and colors) is controlled as a hyper-parameter. Clusters were randomly initialized in thisimplementation.

4. Total Variance Minimization (TVM): a techniquethat reconstructs an image using a randomly selectedsubset of the image pixels with minimal and localizedperturbations. Total variance of the fine-scale imageis measured to remove excessive, namely adversarial,perturbations. We apply pixel dropout by sampling thepixels using a Bernoulli distribution and use the Breg-man method to minimize total variation based on [8].

5. Vector-Quantized Variational Autoencoder (VQ-VAE): is a variant of variational autoencoders that usesdiscrete latent variables. We use the a forward passthrough the encoder and decoder as a lossy compres-sion mechanism. In some of our methods, we train theVQ-VAE with Gaussian noise on the inputs.We then tested and evaluated these defenses over a

range of hyperparameters. Our final accuracy results werebased on the hyperparameters given below:

5Code available at a private Repository: https://github.com/mratan1/cs229_final_proj

6Please email mratan@stanford.edu for access

Defense ParametersJPEG Comp. Quality: 90%

Gauss. Smooth. σx : 0σy : 0.3

K-Means centroids (MNIST): 16centroids (CIFAR-10): 50

Total Var. Min. dropout rate: 0.5weight: 0.03

VQ-VAE hidden size: 256k: 512

batch size: 128epochs: 100α : 2e− 4β: 1

noise: [0, 0.25, 0.75]

To evaluate the robustness of our image transforma-tions, we tested them against four state-of-the-art adver-sarial attacks. Each of these attacks uses some function totransform a normal input into an adversarial one. 7 Theseare:

1. Fast Gradient Sign Method (FGSM): adds noise inthe same direction of the cost function gradient from[7]x+ εsgn(∇xL(θ, x, y));

2. Projected Gradient Descent (PGD): finds perturba-tions focused on gradients that maximize loss from [13]xt+1 = πx+S(xt + αsgn(∇xL(θ, x, y)));

3. CarliniWagnerL2Attack: applies an L2 penalty from[3]min || 12 (tanh(w) + 1)− 1||22 + cf( 1

2 (tanh(w) + 1));

4. DeepFool Attack: projects onto decision boundariesfrom [14]argminr||r||2 such that ∃k : wT

k (x0 + r) + bk ≥wT

k̂(x0)(x0 + r) + bk̂(x0)

.

Based on earlier works, We use max ε = 0.3, whichhas previously allowed for the most powerful attacks.

Examples of these attacks are shown in Figure 1 forFGSM (See Figure 1 for original and Figure 4for at-tacks). Qualitatively, FGSM, PGD, and CarliniWagner2had minimal perturbations to the visible eye. Meanwhile,DeepFool generated noticeably different with the least re-alistic outputs.

For VQ-VAE, we first pre-train the model on the recon-struction task. The weights to the model are then frozenand the encoder / decoder is used as a transformation forthe adversarial image.

Results and DiscussionAfter applying our defenses, we evaluated their model ac-curacies against the original and adversarial accuracies onthe test dataset for MNIST and CIFAR. We summarizeour results in Table 6. For each cell, we describe the ac-curacy we obtained on a given combination of attack anddefense with MNIST’s accuracy on the top, and CIFAR’saccuracy on the same combination on the bottom.

7We use the Foolbox [16] implementation for our work

3

Figure 4: Example of different attacks. DeepFool pro-duced the most noticeable attack.

All the adversarial attacks were effective against theundefended models, achieving a degradation of 0.00% ac-curacy on both datasets (Column 1 in Figure 6). Amongthe attack methods, DeepFool was the most resilientagainst our defenses, in which our defenses had the low-est improvements in accuracy. All defenses had improve-ments less than 10% accuracy. Interestingly, visual in-spection showed that DeepFool also produced the leastrealistic images. It could be possible that it was moredifficult for the compression techniques to remove suchlarge perturbations and distortions. On the other hand,PGD was the easiest attack to overcome. Qualitatively,the images produced are the most realistic. In many cases,the adversarial perturbations were not visible. Since theywere not as visually dominant, the compression algo-rithms could have better compressed and removed theseperturbations.

Despite substantial differences between original andadversarial examples after transformation, all defensesimproved the accuracy of the classifiers (See 6). Theyappeared to minimize perturbations and preserved usefulfeatures. Among the methods, TVM and JPEG performedthe best followed by K-Means. JPEG had the highest ac-curacies for most of the attacks on MNIST, achieving a99% accuracy for FGSM, PGD, and CarliniWagnerL2.Meanwhile, TVM had the highest accuracies on CIFAR,achieving over 70% on FGSM, PGD, and CarliniWagner.Excluding DeepFool, TVM model achieved 70% accu-racy against attacks on both MNIST / CIFAR, which isless than a 30% difference from the original model. Thisshows that TVM and JPEG are both strong simple de-fenses. K-Means was also able to improve accuracies ex-ceeding 50% for both MNIST. More complex defensessuch as Gaussian Smoothing and VQ-VAE did not per-form nearly as well.

Both from these results and qualitative inspections, itseems that the traditional transformations were able toeffectively compress and remove noise from adversarialperturbations, like FGSM, PGD, and CarliniWagnerL2.Since their pertubations appeared small, and TVM, JPEG,and K-means compress to smaller subspaces this likelycleaned the noise. However, DeepFool had larger vari-ance, which could make it more difficult to recover a sim-ilar output to the original.

Between MNIST and CIFAR datasets, we found that

the defenses performed much better in the MINSTdataset. This could possibly be due to the relative com-plexity of the CIFAR dataset. MNIST was smaller in di-mensions, had a smaller range in colors, and had moresimilar qualtitative content. For some adversarial attackswith visible changes, the new output images could appearas different digits.

Comparing the model predictions between the de-fenses, the confusion matrices showed that simpler meth-ods, like JPEG, tended to mispredict a class for specificrespective classes (eg. bird for cat). However, VQ-VAEwas more uniformly dispersed across the classes, possiblydue to its blurring effects. Confusion matrices for JPEGand VQ-VAE are shown (See Figure 7).

Regarding the hyperparameters, we also performed ba-sic ablations studies on some of the models to understandtheir sensitivity to hyperparameters. In particular, adjust-ing the quality of the JPEG compression did not changethe accuracy significantly. On the other hand, the numberof clusters of K-means was sensitive to tuning. For bothMNIST and CIFAR-10, a low number of centroid clusters(eg. n=4 out of 28) resulted in the resulting image beingtoo compressed and losing rich features that the classi-fier requires. On the other hand, a very large number ofcentroid clusters (eg. n=27 out of 28) resulted in minimalcompression of adversarial noise and therefore low accu-racy.

Applying a neural-network based approach, VQ-VAEwas also investigated as a possible new defense. However,we found that it yielded the lowest accuracies across thedifferent defense transformations (See columns 6-8 in 6).While it had one of the highest accuracies on CIFAR forDeep Fool, this was below 10%. It was not effective outof the box.

While adding noise to the VQ-VAE training was ef-fective in improving the results for the MNIST dataset,but it did not improve the CIFAR-10 dataset. In particu-lar, we added Gaussian noise with standard deviations [0,0.25, 0.75] to the input image during VQ-VAE trainingfor the reconstruction task. This resulted in much moreblurred reconstructions and a higher reconstruction loss,but increased object classification accuracy for MNIST(See 5). Increasing the amount of noise however was noteffective at cleaning out noise for the CIFAR-10 dataset.Furthermore, increasing the noise past 1.0 did not resultin improved accuracy on either MNIST or CIFAR andonly made reconstruction worse. We also attempted totune the hyper-parameters for VQ-VAE by performinga coarse grid search of the latent space and hidden sizebut this did not result in an improved accuracy. We findthat the resulting ”cleaned” image, seen in Figure 5 inthe VQ-VAE adversarial example, was significantly moreblurry than the training images. We hypothesize that thisdistributional shift, which was more apparent in the CI-FAR dataset, is the likely cause of the poor performanceof the resulting model. While retraining the model withthe blurred images will likely increase its accuracy, thisis outside the scope of this project. Furthermore, the de-fense becomes attack-specific which is not the goal of thedefensive transformations. On the other hand, one limi-

4

Figure 5: (Left) Examples of image transformations comparing the original, FGSM attack, and their noise differences.Noise on the undefended example appears small to the visible eye.Figure 6: (Right) Predictions on the test datasets for (MNIST, CIFAR) against the undefended model, JPEG compression,Gaussian Smoothing, K-Means, TVM, and VQ-VAE. Bolded accuracies are the highest accuracy for that given attack anddataset. JPEG had most highest accuracies on MNIST while TVM had the highest accuracies on CIFAR.

Figure 7: (Left to right): Predicted accuracy for VQ-VAE, accuracy with adversarial attacks for VQ-VAE, accuracy withadversarial attacks for JPEG, and predicted accuracy for JPEG. We see that overall VQVAE has a much more noticeableincorrect labeling than does JPEG, and that this is distributed across labels.

tation of neural network based defenses is that an adver-sary can use the white box model on the overall (defense+ classifier) model to optimize an attack. To the best ofour knowledge there is no known solution to this problemyet and it remains an open area of research. This blur-ring issue was not observed in the misclassifications ofthe classifier following the JPEG transformation. Uponvisual inspection, we found that these misclassificationsoften did not look realistic or had boundary values (0 or255) in various areas of the image, for VQ-VAE examplein 5. Upon visual inspection, many of these images con-tained salt-and-pepper-like noise which introduced arti-facts to the image that are not captured in the training setof our classifier, and therefore is erroneously classified.

Conclusion and Future WorkIn conclusion, with the growing proliferation of objectclassification models in the world and the increasing ca-pability of these adversarial attacks to deceive state-of-the-art classification models, the importance of adversar-ial defense is growing. In our project, we explored theuse of various traditional and deep-learning based im-age transformation techniques for adversarial image de-fense on state-of-the-art attacks. Regarding novelty, weexplored a new potential defense (VQ-VAE) and evalu-ated defenses on various datasets and attacks. Overall,all five defense transformations improved accuracy. Wefound that the TVM and JPEG image transformations

were the most effective image cleaning methods. Thesimplicity of these models and their model-agnostic na-ture as well as their non-differentiability make them pow-erful and simple first line of defense against adversarialattacks.

For future work, we intend to extend this project in sev-eral ways. In particular, we intend to (1) explore the effec-tiveness of the models on larger datasets such as Imagenet/ CELEB-A, (2) explore combinations of various methodsusing model ensembling (eg. total variance minimizationand GANs), (3) exploring other newer methods of defensesuch as randomly initialized models and bayesian neuralnetworks and convex optimization, (4) training VQ-VAEwith different kinds of noises (eg. saltand-pepper, gaus-sian, uniform). One other potential area to explore wouldbe to increase robustness against structured and targtedperturbations, e.g. graffiti on walls or paint chipping onsigns.

ContributionsAll project aspects were distributed among members.Matthew set-up infrastructure, implemented and testedmany of the models/algorithms, devised experiments andvisualizations, and contributed to write-up. Kimberly pro-vided background research and analysis, implementedand tested models/algorithms, ran experiments and cre-ated visualizations, and contributed to write-up. Nick im-plemented model, provided code support, and contributed

5

to write-up.

References[1] Andrew Kurakin, I. G., and Bengio, S. 2019.

On evaluating adversarial robustness. arXiv preprintarxiv:1902.06705v2.

[2] Athalye, A.; Carlini, N.; and Wagner, D. 2018. Ob-fuscated gradients give a false sense of security: Cir-cumventing defenses to adversarial examples. arXivpreprint arXiv:1802.00420.

[3] Carlini, N., and Wagner, D. 2017. Towards evaluatingthe robustness of neural networks. In 2017 IEEE Sym-posium on Security and Privacy (SP), 39–57. IEEE.

[4] Demontis, A.; Melis, M.; Pintor, M.; Jagielski, M.;Biggio, B.; Oprea, A.; Nita-Rotaru, C.; and Roli, F.2019. Why do adversarial attacks transfer? explain-ing transferability of evasion and poisoning attacks. In28th {USENIX} Security Symposium ({USENIX} Se-curity 19), 321–338.

[5] Finlayson, S. G.; Bowers, J. D.; Ito, J.; Zittrain, J. L.;Beam, A. L.; and Kohane, I. S. 2019. Adversar-ial attacks on medical machine learning. Science363(6433):1287–1289.

[6] Gal, Y., and Smith, L. 2018. Sufficient conditionsfor idealised models to have no adversarial examples:a theoretical and empirical study with bayesian neuralnetworks. arXiv preprint arXiv:1806.00667.

[7] Goodfellow, I. J.; Shlens, J.; and Szegedy, C.2014. Explaining and harnessing adversarial exam-ples. arXiv preprint arXiv:1412.6572.

[8] Guo, C.; Rana, M.; Cisse, M.; and Van Der Maaten,L. 2017. Countering adversarial images using inputtransformations. arXiv preprint arXiv:1711.00117.

[9] Jia, X.; Wei, X.; Cao, X.; and Foroosh, H. 2019.Comdefend: An efficient image compression modelto defend adversarial examples. In Proceedings ofthe IEEE Conference on Computer Vision and PatternRecognition, 6084–6092.

[10] Kingma, D. P., and Welling, M. 2013. Auto-encoding variational bayes. arXiv preprintarXiv:1312.6114.

[11] Kurakin, A.; Goodfellow, I.; and Bengio, S. 2016.Adversarial machine learning at scale. arXiv preprintarXiv:1611.01236.

[12] Luo, Y., and Pfister, H. 2018. Adversarial defense ofimage classification using a variational auto-encoder.arXiv preprint arXiv:1812.02891.

[13] Madry, A.; Makelov, A.; Schmidt, L.; Tsipras, D.;and Vladu, A. 2017. Towards deep learning mod-els resistant to adversarial attacks. arXiv preprintarXiv:1706.06083.

[14] Moosavi-Dezfooli, S.-M.; Fawzi, A.; and Frossard,P. 2016. Deepfool: a simple and accurate method tofool deep neural networks. In Proceedings of the IEEEconference on computer vision and pattern recogni-tion, 2574–2582.

[15] Mozaffari-Kermani, M.; Sur-Kolay, S.; Raghu-nathan, A.; and Jha, N. K. 2014. Systematic poisoningattacks on and defenses for machine learning in health-care. IEEE journal of biomedical and health informat-ics 19(6):1893–1905.

[16] Rauber, J.; Brendel, W.; and Bethge, M. 2017.Foolbox: A python toolbox to benchmark the robust-ness of machine learning models. arXiv preprintarXiv:1707.04131.

[17] Shin, R., and Song, D. 2017. Jpeg-resistant adver-sarial images. In NIPS 2017 Workshop on MachineLearning and Computer Security.

[18] Sitawarin, C., and Wagner, D. 2019a. Defendingagainst adversarial examples with k-nearest neighbor.arXiv preprint arXiv:1906.09525.

[19] Sitawarin, C., and Wagner, D. 2019b. On the ro-bustness of deep k-nearest neighbors. arXiv preprintarXiv:1903.08333.

[20] Su, J.; Vargas, D. V.; and Sakurai, K. 2019. Onepixel attack for fooling deep neural networks. IEEETransactions on Evolutionary Computation.

[21] van den Oord, A.; Vinyals, O.; et al. 2017. Neuraldiscrete representation learning. In Advances in NeuralInformation Processing Systems, 6306–6315.

[22] Wong, E., and Kolter, J. Z. 2017. Provable defensesagainst adversarial examples via the convex outer ad-versarial polytope. arXiv preprint arXiv:1711.00851.

[23] Wong, E.; Schmidt, F.; Metzen, J. H.; and Kolter,J. Z. 2018. Scaling provable adversarial defenses. InAdvances in Neural Information Processing Systems,8400–8409.

6