-
new_empty是获取一个同样device和同样dtype的tensor，bernoulli_(p)是根据概率p产生0,1序列(p是0)，expand_as扩展到同一维度。
def drop_tokens(embeddings, word_dropout):
batch, length, size = embeddings.size()
mask = embeddings.new_empty(batch, length)
mask = mask.bernoulli_(1 - word_dropout)
embeddings = embeddings * mask.unsqueeze(-1).expand_as(embeddings).float()
return embeddings, mask
-
sequence_mask，tensor.ltletter than所有小于的是true。
