- teacher forcing
What is Teacher Forcing for Recurrent Neural Networks?
NOTE:
1⃣️Why?
when in a sequence prediction problem, ways like output-as-input process can be used when training the model, but it can result in problems such as:
- Slow convergence.
- Model instability.
- Poor skill.
2⃣️What is Teacher Forcing?
Teacher forcing works by using the actual or expected output from the training dataset at the current time step y(t) as input in the next time step X(t+1), rather than the output generated by the network.
3⃣️Extension: how to deal with the open loop problem?
beam search
curriculum learning(randomly choosing to use the ground truth output or the generated output from the previous time step as input for the current time step.)
2.应该结合seq2seq的论文一起看：
Sequence to Sequence Learning with Neural Networks
