Building an auto-encoder using Keras

Michael Zippo 18.07.2021

This article will demonstrate the process of compressing data and recovering encoded data using machine learning by first building an auto-encoder using Keras, and then recovering the encoded data and visualizing the recovery. We will be using the MNIST handwritten number set which is pre-loaded into the Keras module, about which you can read here .

The code is structured as follows: first, all the utility functions that are needed at different stages of building the autoencoder are defined, and then each function is called accordingly.

Step 1: Import the required libraries

import numpy as np

import matplotlib.pyplot as plt

from random import randint

from keras import backend as K

from keras.layers import Input , Dense, Conv2D, MaxPooling2D, UpSampling2D

from keras.models import Model

from keras.datasets import mnist

from keras.callbacks import TensorBoard

Step 2: Define a utility function for loading data

       def   load_data (): 
     # determine the size of the input image  
   input_image   =   Input   (shape   =   (  28  ,   28  ,   1  )) 
   
   # Loading data and dividing data into training and test suites  
     (X_train, _), (X_test, _)   =   mnist.load_data () 
    
     # Clean and modify data to fit the model  
     X_train   =   X_train.astype (  ’float32 ’ )   /   255.  
   X_train   =   np .reshape (X_train, (  len   (X_train),   28  ,   28  ,   1  )) 
   X_test   =   X_test.astype (  ’float32’  )   /   255.  
   X_test   =   np.reshape (X_test, (  len   (X_test),   28  ,   28  ,   1  ))  
   
   return   X_train, X_test, input_image

Note. When loading data, note that the space into which the training labels are loaded remains empty because no output labels are used during the compression process.

Step 3: Defining a utility function for building the Auto-encoder neural network

def build_network (input_image):

# Build the auto-encoder encoder

x = Conv2D ( 16 , ( 3 , 3 ), activation = ’relu’ , padding = ’same’ ) (input_image)

x = MaxPooling2D (( 2 , 2 ), padding = ’ same’ ) (x)

x = Conv2D ( 8 , ( 3 , 3 ), activation = ’relu’ , padding = ’same’ ) (x)

x = MaxPooling2D (( 2 , 2 ), padding = ’same’ ) (x)

x = Conv2D ( 8 , ( 3 , 3 ), activation = ’relu’ , padding = ’same’ ) (x)

encoded_layer = MaxPooling2D (( 2 , 2 ), padding = ’ same’ ) (x)

# Build auto-encoder decoder

x = Conv2D ( 8 , ( 3 , 3 ), activation = ’relu’ , padding = ’same’ ) (encoded_layer)

x = UpSampling2D (( 2 , 2 )) (x)

x = Conv2D ( 8 , ( 3 , 3 ), activation = ’ relu’ , padding = ’same’ ) (x)

x = UpSampling2D (( 2 , 2 )) (x)

x = Conv2D ( 16 , ( 3 , 3 ), activation = ’relu’ ) (x)

x = UpSampling2D (( 2 , 2 )) (x)

decoded_layer = Conv2D ( 1 , ( 3 , 3 ), activation = ’sigmoid’ , padding = ’same’ ) (x)

return decoded_layer

Step 4: Define a utility function to build and train an auto-encoder network

def build_auto_encoder_model (X_train, X_test, input_image, decoded_layer):

# Define auto-encoder parameters

autoencoder = Model (input_image, decoded_layer)

autoencoder. compile (optimizer = ’adadelta’ , loss = ’binary_crossentropy’ )

# Auto-encoder training

autoencoder.fit (X_train, X_train ,

epochs = 15 ,

batch_size = 256 ,

shuffle = True ,

validation_data = (X_test, X_test),

callbacks = [TensorBoard (log_dir = ’/ tmp / autoencoder’ )])

return autoencoder

Step 5: Defining a utility function for rendering a reconstruction

def visualize (model, X_test):

# Recover encoded images

reconstructed_images = model.predict (X_test)

plt.figure (figsize = ( 20 , 4 ))

for i in range ( 1 , 11 ):

   # Generate random to get random results  
   rand_num   =   randint (  0  ,   10001  ) 
   
   # To display the original image  
     ax   =   plt.subplot (  2  ,   10  , i)