Loading a trained Keras model and continue training

👻 Check our latest review to choose the best laptop for Machine Learning engineers and Deep learning tasks!

I was wondering if it was possible to save a partly trained Keras model and continue the training after loading the model again.

The reason for this is that I will have more training data in the future and I do not want to retrain the whole model again.

The functions which I am using are:

#Partly train model
model.fit(first_training, first_classes, batch_size=32, nb_epoch=20)

#Save partly trained model
model.save("partly_trained.h5")

#Load partly trained model
from keras.models import load_model
model = load_model("partly_trained.h5")

#Continue training
model.fit(second_training, second_classes, batch_size=32, nb_epoch=20)

Edit 1: added fully working example

With the first dataset after 10 epochs the loss of the last epoch will be 0.0748 and the accuracy 0.9863.

After saving, deleting and reloading the model the loss and accuracy of the model trained on the second dataset will be 0.1711 and 0.9504 respectively.

Is this caused by the new training data or by a completely re-trained model?

"""
Model by: http://machinelearningmastery.com/
"""
# load (downloaded if needed) the MNIST dataset
import numpy
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import np_utils
from keras.models import load_model
numpy.random.seed(7)

def baseline_model():
    model = Sequential()
    model.add(Dense(num_pixels, input_dim=num_pixels, init="normal", activation="relu"))
    model.add(Dense(num_classes, init="normal", activation="softmax"))
    model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
    return model

if __name__ == "__main__":
    # load data
    (X_train, y_train), (X_test, y_test) = mnist.load_data()

    # flatten 28*28 images to a 784 vector for each image
    num_pixels = X_train.shape[1] * X_train.shape[2]
    X_train = X_train.reshape(X_train.shape[0], num_pixels).astype("float32")
    X_test = X_test.reshape(X_test.shape[0], num_pixels).astype("float32")
    # normalize inputs from 0-255 to 0-1
    X_train = X_train / 255
    X_test = X_test / 255
    # one hot encode outputs
    y_train = np_utils.to_categorical(y_train)
    y_test = np_utils.to_categorical(y_test)
    num_classes = y_test.shape[1]

    # build the model
    model = baseline_model()

    #Partly train model
    dataset1_x = X_train[:3000]
    dataset1_y = y_train[:3000]
    model.fit(dataset1_x, dataset1_y, nb_epoch=10, batch_size=200, verbose=2)

    # Final evaluation of the model
    scores = model.evaluate(X_test, y_test, verbose=0)
    print("Baseline Error: %.2f%%" % (100-scores[1]*100))

    #Save partly trained model
    model.save("partly_trained.h5")
    del model

    #Reload model
    model = load_model("partly_trained.h5")

    #Continue training
    dataset2_x = X_train[3000:]
    dataset2_y = y_train[3000:]
    model.fit(dataset2_x, dataset2_y, nb_epoch=10, batch_size=200, verbose=2)
    scores = model.evaluate(X_test, y_test, verbose=0)
    print("Baseline Error: %.2f%%" % (100-scores[1]*100))

Edit 2: tensorflow.keras remarks

For tensorflow.keras change the parameter nb_epochs to epochs in the model fit. The imports and basemodel function are:

import numpy
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.models import load_model


numpy.random.seed(7)

def baseline_model():
    model = Sequential()
    model.add(Dense(num_pixels, input_dim=num_pixels, activation="relu"))
    model.add(Dense(num_classes, activation="softmax"))
    model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
    return model

👻 Read also: what is the best laptop for engineering students?

Loading a trained Keras model and continue training __del__: Questions

import time
time.sleep(5)   # Delays for 5 seconds. You can also use a float value.

import time
while True:
    print("This prints once a minute.")
    time.sleep(60) # Delay for 1 minute (60 seconds).

from time import sleep
sleep(0.1) # Time in seconds

Loading a trained Keras model and continue training __del__: Questions

We hope this article has helped you to resolve the problem. Apart from Loading a trained Keras model and continue training, check other __del__-related topics.

Want to excel in Python? See our review of the best Python online courses 2023. If you are interested in Data Science, check also how to learn programming in R.

By the way, this material is also available in other languages:

• Italiano Loading a trained Keras model and continue training
• Deutsch Loading a trained Keras model and continue training
• Français Loading a trained Keras model and continue training
• Español Loading a trained Keras model and continue training
• Türk Loading a trained Keras model and continue training
• Русский Loading a trained Keras model and continue training
• Português Loading a trained Keras model and continue training
• Polski Loading a trained Keras model and continue training
• Nederlandse Loading a trained Keras model and continue training
• 中文 Loading a trained Keras model and continue training
• 한국어 Loading a trained Keras model and continue training
• 日本語 Loading a trained Keras model and continue training
• हिन्दी Loading a trained Keras model and continue training