在画神经网络的模型图，遇到了一个问题，LSTM模型的输出到底是什么？
由于用到的Toolkit是Keras，因此关键是对Keras中的两个参数进行理解。
1.return_seqeunces
return_sequences:默认为false。当为false时，返回最后一层最后一个步长的hidden state;当为true时，返回最后一层的所有hidden state。
2. return_state
return_state:默认false.当为true时，返回最后一层的最后一个步长的输出hidden state和输入cell state。
3. 举例说明
下图的输入是一个步长为3,维度为1的数组。
一共有2层神经网络（其中第一层必须加上“return_sequences=True”,这样才能转化成步长为3的输入变量）
（1）return_sequences=True
from keras.models import Model
from keras.layers import Input
from keras.layers import LSTM
from numpy import array
from keras.models import Sequential
data = array([0.1,0.2,0.3]).reshape((1,3,1))
inputs1 = Input(shape=(3,1))
lstm1,state_h,state_c = LSTM(2,return_sequences=True,return_state=True)(inputs1)
lstm2 = LSTM(2,return_sequences=True)(lstm1)
model = Model(input = inputs1,outputs = [lstm2])
print(model.predict(data))
输出结果为：（最后一层lstm2的每一个时间步长hidden_state的结果）
[[[0.00120299 0.0009285 ]
[0.0040868 0.00327 ]
[0.00869473 0.00720878]]]
(2)return_sequence = False , return_state = True
from keras.models import Model
from keras.layers import Input
from keras.layers import LSTM
from numpy import array
from keras.models import Sequential
data = array([0.1,0.2,0.3]).reshape((1,3,1))
inputs1 = Input(shape=(3,1))
lstm1,state_h,state_c = LSTM(2,return_sequences=True,return_state=True)(inputs1)
lstm2,state_h2,state_c2 = LSTM(2,return_state=True)(lstm1)
model = Model(input = inputs1,outputs = [lstm2,state_h2,state_c2])
print(model.predict(data))
输出为：
因为return_state=True，返回了最后一层最后一个时间步长的输出hidden_state和输入cell_state.
[array([[-0.00234587, 0.00718377]], dtype=float32), array([[-0.00234587, 0.00718377]], dtype=float32), array([[-0.00476015, 0.01406127]], dtype=float32)]
(3)return_sequence = True , return_state = True
from keras.models import Model
from keras.layers import Input
from keras.layers import LSTM
from numpy import array
from keras.models import Sequential
data = array([0.1,0.2,0.3]).reshape((1,3,1))
inputs1 = Input(shape=(3,1))
lstm1,state_h,state_c = LSTM(2,return_sequences=True,return_state=True)(inputs1)
lstm2,state_h2,state_c2 = LSTM(2,return_sequences=True,return_state=True)(lstm1)
model = Model(input = inputs1,outputs = [lstm2,state_h2,state_c2])
print(model.predict(data))
输出为：最后一层所有时间步长的hidden state,及最后一层最后一步的hidden state,cell state.
[array([[[-2.0248523e-04, -1.0290105e-03],
[-3.6455912e-04, -3.3424206e-03],
[-3.6696041e-05, -6.6624139e-03]]], dtype=float32),
array([[-3.669604e-05, -6.662414e-03]], dtype=float32),
array([[-7.3107367e-05, -1.3788906e-02]], dtype=float32)]
(4)return_sequence = False , return_state = False
from keras.models import Model
from keras.layers import Input
from keras.layers import LSTM
from numpy import array
from keras.models import Sequential
data = array([0.1,0.2,0.3]).reshape((1,3,1))
inputs1 = Input(shape=(3,1))
lstm1,state_h,state_c = LSTM(2,return_sequences=True,return_state=True)(inputs1)
lstm2 = LSTM(2)(lstm1)
model = Model(input = inputs1,outputs = [lstm2])
print(model.predict(data))
输出为：最后一层的最后一个步长的hidden state.
[[-0.01998264 -0.00451741]]
上述参考：
https://www.wandouip.com/t5i152855/
那么对于RNN和LSTM的outputs和hidden state应该有更进一步的理解，可以参考：
https://zhuanlan.zhihu.com/p/28919765
