fix RNN and IfElse syntax in Block design#4210
fix RNN and IfElse syntax in Block design#4210wangkuiyi merged 16 commits intoPaddlePaddle:developfrom
Conversation
doc/design/block.md
Outdated
| # mark the variables that need to be segmented for time steps. | ||
| x_ = x.as_step_input() | ||
| # mark the varialbe that used as a RNN state. | ||
| h_ = h.as_step_memory(init=m) |
doc/design/block.md
Outdated
| x = ie.inputs(true, 0) | ||
| z = operator.add(x, y) | ||
| ie.set_output(true, 0, operator.softmax(z)) | ||
| x_ = x.as_ifelse_input() |
There was a problem hiding this comment.
If the framework knows whether x contains instances, user does not indicate x as ifelse_input. The use can directly use x in the block and the framework can automatically do the splitting.
There was a problem hiding this comment.
some x that has instances might not need to be split.
such as
# v is some op's output
v = some_op() # shape is [20, 20]
data = var(shape=[20, 20])
with ie.true_block():
# split x
x = data.as_ifelse_input() # shape [1, 20]
# v should not be split
y = pd.matmul(x.T, v) # shape [1, 20]
y_T = y.T # [20, 1]
ie.set_outputs(y_T)v has the same batch_size, but do not need to be split.
if write as
with ie.true_block():
y = pd.matmul(x, v) # shape [1, 20] x [1, 20] wrong
the shapes will not match.
There was a problem hiding this comment.
If a variable has fixed size, it's not splittable. If a variable's size depends on batchsize, it must be splitted because it means that it contains data for each instances.
There was a problem hiding this comment.
oh, I see, I will change this latter.
| ie = pd.ifelse_builder() | ||
|
|
||
| with ie.true_block(): | ||
| x = ie.inputs(true, 0) |
| with rnn.step(): | ||
| h = rnn.memory(init = m) | ||
| hh = rnn.previous_memory(h) | ||
| a = layer.fc(W, x) |
There was a problem hiding this comment.
RNN needs to differentiate static input and sequence input. Static input is same for every step. Sequence input will have to pick the corresponding data for each step. I suggest that static input uses same syntax as if-else, while sequence input needs to explicitly indicate it as step input (e.g., using as_step_input())
3 participants
No description provided.