ML | Transferring Learning with Convolutional Neural Networks

Python Methods and Functions

ResNet Introduction
ResNet was originally developed as a method to solve the vanishing gradient problem. This is a problem where backpropagating gradients become extremely small as they multiply over and over again, limiting the size of the neural network. The ResNet architecture tries to solve this problem by skipping connections, that is, adding shortcuts that allow data to skip past layers.

The model consists of a series of convolutional layers + skipped connections, then a middle pool, then an output fully connected (dense) layer. For transfer learning, we want the convolutional layers to contain only the features we are interested in, so we would like to skip them when importing the model. Finally, since we are deleting the output layers, we need to replace them with our own series of layers.

Statement of the problem
To illustrate the process of teaching transmission, I will use Caltech-101 dataset, image dataset with 101 categories and approximately 40-800 images per category.

Data Processing

First download and extract the dataset here . Be sure to delete the "BACKGROUND_Google" folder after checkout.

Code: For proper evaluation, we need to separate the data into training and test cases. Here we need to split within each category to ensure proper presentation in the test suite.

TEST_SPLIT = 0.2

VALIDATION_SPLIT = 0.2

 

import os

import math

 
# stores test data

os.mkdir ( " caltech_test "

  

for cat in os.listdir ( "101_ObjectCategories /" ):

# moves the x portion of the images in each category to test images

  # new category folder

  os.mkdir ( " caltech_test / " + cat) 

imgs = os.listdir ( "101_ObjectCategories /" + cat) 

  # all image file names

split = math.floor ( len ( imgs) * TEST_SPLIT) 

test_imgs = imgs [: split]

# move test part

for t_img in test_imgs : 

os.rename ( " 101_ObjectCategories / " + cat + "/" + t_img, 

  "caltech_test /" + cat + " / " + t_img)

Exit:

 This above code creates the file structure: 101_ObjectCategories / - accordion - airplanes - anchor - ... caltech_test / - accordion - airplanes - anchor - ... 

The first folder contains images of trains, the second contains test images. Each subfolder contains images belonging to this category. We will use the Keras ImageDataGenerator class to enter data. ImageDataGenerator allows for easy processing of image data and also has options for enlargement.

# make sure the preprocessing function of the original model matches

from keras.applications.resnet50 import preprocess_input 

from keras.preprocessing.image import ImageDataGenerator

 

train_gen = ImageDataGenerator (

validation_split = 0.2

preprocessing_function = preprocess_input)

train_flow = train_gen.flow_from_directory ( "101_ObjectCategories /"

target_size = ( 256 , 256 ), 

batch_size = 32

  subset = "training" )

 

valid_flow = train_gen.flow_from_directory ( "101_ObjectCategories /"

target_size = ( 256 , 256 ), 

  batch_size = 32

subset = "validation" )

 

test_gen = ImageDataGenerator (

  preprocessing_function = preprocess_input)

test_flow = test_gen.flow_from_directory ( "caltech_test"

target_size = ( 256 , 256 ), 

  batch_size = 32 )

The above code takes the path to the image directory file and creates an object for generating data.

Building the model
Code: add a basic pretrained model.

from keras.applications.resnet50 import ResNet50

from keras.layers import GlobalAveragePooling2D, Dense

from keras.layers import BatchNormalization, Dropout

from keras.models import Model

 
# default zag the rooted model will include the original CNN
# the classifier is for the ImageNet dataset
# as we want use this model for another problem,
# we need to omit the original fully linked layers, and
# replace them with our own setting include_top = False will
# load model without fully linked layer

 
# reload model with pre-prepared imagenet weights.

res = ResNet50 (weights = ' imagenet' , include_top = False

input_shape = ( 256 , 256 , 3 )) 

This dataset is relatively small — about 5628 images after splitting, with most categories only having 50 images, so fine-tuning the convolutional layers can lead to overfitting. Our new dataset is very similar to the ImageNet dataset, so we can be sure that many of the pretrained weights have the correct characteristics as well. This way we can freeze these trained convolutional layers so that they don't change when we train the rest of the classifier. If you have a smaller dataset that is significantly different from the original, fine-tuning can still lead to overfitting, but later layers will not contain the correct functionality. This way, you can freeze the convolutional layers again, but only use the results of the earlier layers as they contain more general functionality. With a large dataset, you don't have to worry about overfitting, so you can often fine tune the entire network.

from keras.applications.resnet50 import ResNet50

from keras.layers import GlobalAveragePooling2D, Dense

from keras.layers import BatchNormalization, Dropout

from keras.models import Model

 
# by default the loaded model will be include original CNN
# classifier is for ImageNet dataset
# since we want to use this model for another problem,
# we need to omit the original fully linked layers, and
# replace them with our own setting include_top = False will
# load model without fully linked layer

 
# reload model with pre-prepared imagenet weights.

res = ResNet50 (weights = ' imagenet' , include_top = False

  input_shape = ( 256 , 256 , 3 )) 

Now we can add the rest of the classifier. This takes the output from the pretrained convolutional layers and injects it into a separate classifier that trains on the new dataset.

# get output from loaded model

x = res .output 

 
# cf. pools by spatial dimensions (rows, columns)
# until zero. Converts the data to 1D, allowing
# for correct typing into dense layers
# (for example, (8, 8, 2048) - & gt; (2048)).

x = GlobalAveragePooling2D () (x) 

 
# subtracts the batch average and divides by the batch standard deviation
# reduce the bias of the input distributions between layers.

x = BatchNormalization () (x) 

 
# dropout allows layers to be less dependent on
# defined features that reduce overfitting

x = Dropout ( 0.5 ) (x) 

x = Dense ( 512 , activation = ' relu' ) (x)

x = BatchNormalization () (x)

x = Dropout ( 0.5 ) (x)

 
# output class layer, we have 101 classes,
# we need 101 output neurons

x = Dense ( 101 , activation = 'softmax' ) (x) 

 
# create model, I / O setup

model = Model (res. input , x) 

 
# compile the model - we train with the Adam Optimizer
# and Categorical cross-entropy as a loss function

model. compile (optimizer = 'Adam'

  loss = 'categorical_crossent ropy'

metrics = [ 'accuracy' ]) 

 
# our model structure
model.summary () 

Code: train the model

model.fit_generator (train_flow, epochs = 5 , validation_data = valid_flow)


Output:

Epoch 1/5 176/176 [==============================] - 27s 156ms / step - loss: 1.6601 - acc : 0.6338 - val_loss: 0.3799 - val_acc: 0.8922 Epoch 2/5 176/176 [=============================] - 19s 107ms / step - loss: 0.4637 - acc: 0.8696 - val_loss: 0.2841 - val_acc: 0.9225 Epoch 3/5 176/176 [====================== ========] - 19s 107ms / step - loss: 0.2777 - acc: 0.9211 - val_loss: 0.2714 - val_acc: 0.9225 Epoch 4/5 176/176 [============ ==================] - 19s 107ms / step - loss: 0.2223 - acc: 0.9327 - val_loss: 0.2419 - val_acc: 0.9284 Epoch 5/5 176/176 [== ===========================] - 19s 106ms / step - loss: 0.1784 - acc: 0.9461 - val_loss: 0.2499 - val_acc: 0.9239 

Code: to evaluate test case

result = model.evaluate (test_flow)

  

print ( 'The model achieved a loss of % .2f and, '

  ' accuracy of% .2f %%. ' % (result [ 0 ], result [ 1 ] * 100 ))

Exit:

 53/53 [==================== =========] - 5s 95ms / step The model achieved a loss of 0.23 and accuracy of 92.80%. 

For class 101 dataset, we achieved 92.8% accuracy in just 5 epochs. For perspective, the original ResNet was trained on a dataset of ~ 1 million images, over 120 epochs.
There are a couple things that can be improved. On the one hand, looking at the mismatch between loss of validation and loss of training in the past era, you can see that the model is starting to retool. One way to solve this problem — add image enlargement. Simple image enlargement can be easily implemented with using the ImageDataGenerator class . You can also play around with adding / removing layers or changing hyper parameters such as dropout or dense layer size.
By executing this code here with Google Colab's free GPU computing resources.





Get Solution for free from DataCamp guru