As described in the class, adversarial examples are a manifestation of metamorphic testing on machine learning modules. This coursework aims to help you generate adversarial examples of your own, both of the
In the first part of this coursework, you will be loosely recreating results from the earlier papers that made adversarial examples, but instead of using their gradient-based optimization schema, you will use a genetic algorithm to find vectors that cause misclassification.
- Goodfellow et al., Explaining and Harnessing Adversarial Examples
- Carlini and Wagner, Towards Evaluating the Robustness of Neural Networks
- Su et al., One pixel attack for fooling deep neural networks
In the second part, you will implement "realistic" perturbations that are nonetheless scalable. The effects are similar to the following paper:
Specifically, in this coursework, you will be generating adversarial examples based on the 'seed' images provided in the images/ directory. In this assignment, an "adversarial example" is defined as an image which is modified from the original image such that the original (correct) label is no longer the most likely prediction according to the model. See the example results in images/examples/.
- Install the Python dependencies via
requirements.txt.⚠️ For maximum accessability, the requirements use the CPU version of PyTorch. If possible, using GPU resources may help your code run faster. The instructions for installing different versions of Pytorch can be found here.
In the first task, you will implement genetic algorithms that find adversarial examples (more specifically, adversarial noise) that differ from the original image by a maximum of
- The
$L_\infty$ norm, which measures the maximum difference between the original pixels and the adversarial example. A good$L_\infty$ adversarial example will change almost all the pixels by a small value. - The
$L_2$ norm, which measures the Euclidean distance between the original image and the adversarial image. - The
$L_0$ norm (actually mathematically not a norm) measures the number of pixels which changed between the original and adversarial image. A good$L_0$ adversarial example would change a few pixels by a large amount.
Implement a GA for each norm, including a fitness function, in epsilon_ball.py. Your task is to generate noise that can be added to the image tensor, so that the neural network (VGG-11) will mispredict the image. The norms themselves are implemented in util.py, so you may make use of them. (The observant of you will notice that e.g. get_pixel_norm_for function if you want to play with that, although it will not be used for evaluation.) Using the get_viz_for function, you may generate an image which visualizes the top ten likely classes and their probabilities according to the VGG-11 model. Look to the images/init_viz/ directory for visualization examples for the original images.
For the first task, you will be evaluated based on the smallest
In the second task, instead of generating meaningless noise, you will use Perlin noise to implement "fog" that in turn induces DNN misprediction. In this scenario, instead of directly manipulating the noise vectors as you did in the first task, you are to evolve the parameters of Perlin noise to induce misclassification. An example of combining Perlin noise is provided on the Perlin noise package webpage, also presented here for completeness:
import matplotlib.pyplot as plt
from perlin_noise import PerlinNoise
noise1 = PerlinNoise(octaves=3)
noise2 = PerlinNoise(octaves=6)
noise3 = PerlinNoise(octaves=12)
noise4 = PerlinNoise(octaves=24)
xpix, ypix = 100, 100
pic = []
for i in range(xpix):
row = []
for j in range(ypix):
noise_val = noise1([i/xpix, j/ypix])
noise_val += 0.5 * noise2([i/xpix, j/ypix])
noise_val += 0.25 * noise3([i/xpix, j/ypix])
noise_val += 0.125 * noise4([i/xpix, j/ypix])
row.append(noise_val)
pic.append(row)For this task, evolve a list of [(octave, magnitude)] parameters that will be added as above; for example, the representation for the noise in the code above would be [(3, 1), (6, 0.5), (12, 0.25), (24, 0.125)].
As we still need some sort of metric to judge how much obscuring the "fog" is doing, the sum of the magnitude parameters is used to evaluate the degree of obscuring. Again, having a "weaker" fog according to this metric and still causing misclassification is the goal.
Based on the contents of your epsilon_ball.py / perlin_fog.py, we will evaluate the epsilon_ball.py on CPU, and five minutes for perlin_fog.py. See grade.py for an example of a grading script. (The actual grading will use the same logic, but will use additional safeguards to prevent tampering of key functions such as the loss function when grading, as well as to deal with timeouts.)
Finally, include a PDF report of your efforts when you are done. In the report, include the following:
- A description of the strategies that you used for each of the norms, and whether you felt compelled to use different strategies in each case
- The 15 images that you evolved for each seed image and norm, their
$\epsilon$ distance from the original image, the 15 raw tensor files generated byget_viz_for(their filenames start withraw_), and the classification results as visualized by theget_viz_forfunction. For each seed image and norm, include the$\epsilon$ distance from the original image in the report. - (Bonus) The evolved Perlin noise expression, and a visualization of the best foggy images generated.