Skip to content

Adversarial sample#7270

Merged
reyoung merged 7 commits intoPaddlePaddle:developfrom
gx-wind:adversarial-sample
Jan 11, 2018
Merged

Adversarial sample#7270
reyoung merged 7 commits intoPaddlePaddle:developfrom
gx-wind:adversarial-sample

Conversation

@gx-wind
Copy link

@gx-wind gx-wind commented Jan 6, 2018

We come from Baidu security lab. We want to contribute the advbox tools which can generate adversarial sample to fool the neural networks like tensorflow's cleverhans library.

@gx-wind gx-wind closed this Jan 6, 2018
@gx-wind gx-wind reopened this Jan 6, 2018
@@ -0,0 +1,9 @@
# Advbox
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/adversarial/ may not be a good path. However, we can just merge this PR, and change it later.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. You can specify an appropriate directory. Thank you.

The base model of the model.
"""
from abc import ABCMeta
#from advbox.base import Model
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either uncomment or remove this line.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line has been deleted.

"""
The base model of the model.
"""
from abc import ABCMeta
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from abc import ABCMeta, abstractmethod
@JiayiFeng JiayiFeng self-assigned this Jan 9, 2018
Copy link
Collaborator

@JiayiFeng JiayiFeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very high-quality code! Many thanks!


Args:
image(numpy.ndarray): image with shape (height, width, channel)
label(int): image label used to cal gradient.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments seem inconsistent with real arguments.

Args:
program(paddle.v2.fluid.framework.Program): The program of the model which generate the adversarial sample.
input_name(string): The name of the input.
logits_name(string): The name of the logits.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the logits_name means the fake label logits to cheat NN?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adversarial attack has two types, untargeted and targeted.

  • The goal of the non-targeted attack is to slightly modify source image in a way that image will be classified incorrectly
  • The goal of the targeted attack is to slightly modify source image in a way that image will be classified as specified target class

In untargeted attack which we has implemented, the fake label logits make no effect.
In targeted attack, the logits_name means the fake label logits to cheat NN.We will implement the targeted attack later.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my understanding, even in the non-targeted attack, the fake label logits are required to generate the loss. Otherwise, we can neither get the backward gradient nor modify input images further. However, what the fake label logits are is not important. They can be all zeros, random vectors or even different each batch.

Do I understand it correctly?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

epsilons = np.linspace(0, 1, num=epsilons + 1)

for epsilon in epsilons:
adv_img = image_batch[0][0].reshape(
Copy link
Collaborator

@JiayiFeng JiayiFeng Jan 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we only get the first instance of the image_batch, so the batch size can only be 1 ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I have changed the name and modify the comment to avoid ambiguity.

loss = self._program.block(0).var(self._cost_name)
param_grads = fluid.backward.append_backward(
loss, parameter_list=[self._input_name])
self._gradient = param_grads[0][1]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that self._gradient should be the gradient of input. However, it is not always in the first element. So the following may be more correct:

self._gradient = dict(param_grads)[self._input_name]
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has fixed.

Returns:
Variable: the label prediction
"""
#conv1 = fluid.nets.conv2d()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the comment.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has removed

# if avg cost less than 10.0 and accuracy is larger than 0.9, we think our code is good.
break

# exit(0)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the comment.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has removed.

fluid.io.save_params(
exe, dirname='./mnist', main_program=fluid.default_main_program())
print('train mnist done')
exit(1)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exit(0)? 1 means there is something wrong.

Actually, I think exit is not necessary here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has removed.

Copy link
Collaborator

@JiayiFeng JiayiFeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Welcome to be a PaddlePaddle contributor!

@reyoung reyoung merged commit 0e54477 into PaddlePaddle:develop Jan 11, 2018
@wangkuiyi
Copy link
Collaborator

I noticed that this PR merges code into the root directory. Let's create an examples directory, and move the code there. I created an issue to remark this #7542

@gx-wind gx-wind deleted the adversarial-sample branch January 16, 2018 06:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

4 participants