Actually LSTM supports three-dimensional input. They are – (samples, time steps, features)
- Samples. One complete sequence is considered as one sample. A batch may contains one or more samples. In NLP, if we are dealing with the text at sentence level (means taking one sentence at a time), then our sample size will be one.
- Time Steps. It can be considered as the number of times, we feed to the model. In the above discussed one-sentence case of NLP, the number of words in the sentence will be equivalent to the time-steps.
- Features. It represents the number of columns of each sample. In the above one sentence case of NLP, we will obtain it through embedding. Generally, we use word2vec, Glove, etc. to get the word embeddings. Thus, in this case the dimension of the embedding will considered as feature size.
Now, we will try to understand, how to reshape 1D and 2D array to 3D Numpy array and apply LSTM on the top of that. We will also see, how LSTM works on 3D Numpy array.
1D Numpy Array with LSTM
# Using 1D array with LSTM from keras import Model from keras.layers import Input, Dense, Bidirectional from keras.layers.recurrent import LSTM import numpy as np # define model for simple BI-LSTM + DNN based binary classifier def define_model(): input1 = Input(shape=(1,1)) #take the reshape last two values, see "data = np.reshape(data,(10,1,1))" which is "data/batch-size, row, column" lstm1 = Bidirectional(LSTM(units=32))(input1) dnn_hidden_layer1 = Dense(3, activation='relu')(lstm1) dnn_output = Dense(1, activation='sigmoid')(dnn_hidden_layer1) model = Model(inputs=[input1],outputs=[dnn_output]) # compile the model model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) model.summary() return model # Take 1-D array data with 10 records/elements data = np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]) Y = [1,1,1,1,1,0,0,0,0,0] #define the label for binary classification print("data = ", data) # Reshape the data to 3D-Numpy array data = np.reshape(data,(10,1,1)) print("data after reshape => ",data) # Call the model model = define_model() # Fit the model model.fit([data],[np.array(Y)],epochs=4,batch_size=2,verbose=1) # Take a test data to test the working of the model test_data = np.array([[0.29]]) # reshape the test data test_data = np.reshape(test_data,(1,1,1)) # predict the sigmoid output [0,1] for the 'test_data' pred = model.predict(test_data) print("predicted sigmoid output => ",pred)
2D Numpy Array with LSTM
# Using 2D array with LSTM from keras import Model from keras.layers import Input, Dense, Bidirectional from keras.layers.recurrent import LSTM import numpy as np # define model for simple BI-LSTM + DNN based binary classifier def define_model(): input1 = Input(shape=(2,1)) #take the reshape last two values, see "data = np.reshape(data,(10,2,1))" which is "data/batch-size, row, column" lstm1 = Bidirectional(LSTM(units=32))(input1) dnn_hidden_layer1 = Dense(3, activation='relu')(lstm1) dnn_output = Dense(1, activation='sigmoid')(dnn_hidden_layer1) model = Model(inputs=[input1],outputs=[dnn_output]) # compile the model model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) model.summary() return model # Take a dummy 2D numpy array to train the model data = np.array([[0.1, 0.15], [0.2, 0.25], [0.3, 0.35], [0.4, 0.45], [0.5, 0.55], [0.6, 0.65], [0.7, 0.75], [0.8, 0.85], [0.9, 0.95], [1.0, 1.5]]) Y = [1,1,1,1,1,0,0,0,0,0] #Class label for the dummy data print("data = ", data) # Reshape the data into 3-D numpy array data = np.reshape(data,(10,2,1)) #Here we have a total of 10 rows or records print("data after reshape => ",data) # Call the model model = define_model() # fit the model model.fit([data],[np.array(Y)],epochs=4,batch_size=2,verbose=1) # Take a test data to test the working of the model test_data = np.array([[0.2, 0.33]]) # reshape the test data test_data = np.reshape(test_data,(1,2,1)) # predict the sigmoid output [0,1] for the 'test_data' pred = model.predict(test_data) print("predicted sigmoid output => ",pred)
3D Numpy Array with LSTM
# Using 3D array with LSTM from keras import Model from keras.layers import Input, Dense, Bidirectional from keras.layers.recurrent import LSTM import numpy as np # define model for simple BI-LSTM + DNN based binary classifier def define_model(): input1 = Input(shape=(2,2)) #use row and column size as input size lstm1 = Bidirectional(LSTM(units=32))(input1) dnn_hidden_layer1 = Dense(3, activation='relu')(lstm1) dnn_output = Dense(1, activation='sigmoid')(dnn_hidden_layer1) model = Model(inputs=[input1],outputs=[dnn_output]) # compile the model model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) model.summary() return model # Take a dummy 3D numpy array to train the model data = np.array([[[0.1, 0.15], [0.2, 0.25]], [[0.3, 0.35], [0.4, 0.45]], [[0.5, 0.55], [0.6, 0.65]], [[0.7, 0.75], [0.8, 0.85]], [[0.9, 0.95], [1.0, 1.5]]]) Y = [1,1,1,0,0] #define binary class level for this model print("data = ", data) # NO NEED TO RESHAPE THE DATA as it is already in 3D format # Call the model model = define_model() # Fit the model model.fit([data],[np.array(Y)],epochs=4,batch_size=2,verbose=1) # Take a test data to test the working of the model test_data = np.array([[[0.2, 0.33],[0.2, 0.33]]]) # predict the sigmoid output [0,1] for the 'test_data' pred = model.predict(test_data) print("predicted sigmoid output => ",pred)