While any implementation of the Asynchronous Advantage Actor Critic algorithm must be complex, all implementations will have one thing in common — the presence of a global network and a working class.
- The Global Network: class contains all the necessary operations Tensorflow for autonomous creation of neural networks.
- Working class. This class is used to simulate the process of training an employee who has his own copy of the environment and a "personal" neural network.
The following implementation will require the following modules:
The following lines of code indicate the basic functionality required to create the corresponding class.
The following lines contain various functions that describe the member functions of the class defined above.
-" tf.placeholder () - Inserts a placeholder for a tensor that will always be fed. -" tf.reshape () - Reshapes the input tensor -" slim.conv2d () - Adds an n-dimensional convolutional network -" slim.fully_connected () - Adds a fully connected layer
Note the following definitions:
- Filter: this is a small matrix that is used to apply various effects to a given image.
- Padding: is the process of adding an extra row or column at the edges of an image to fully compute filter convolution values.
- Step: is the number of steps after which the filter is set to a pixel in the given direction.
Recurrent construction networks:
-" tf.nn.rnn_cell.BasicLSTMCell () - Builds a basic LSTM Recurrent network cell -" tf.expand_dims () - Inserts a dimension of 1 at the dimension index axis of input’s shape -" tf.shape () - returns the shape of the tensor -" tf.nn.rnn_cell.LSTMStateTuple () - Creates a tuple to be used by the LSTM cells for state_size, zero_state and output state. -" tf.nn.dynamic_rnn () - Builds a Recurrent network according to the Recurrent network cell
Generate pricing and policy output layers:
Build a master network and deploy workers:
Performing parallel Tensorflow operations:
-" tf.Session () - A class to run the Tensorflow operations -" tf.train.Coordinator () - Returns a coordinator for the multiple threads -" tf.train.get_checkpoint_state () - Returns a valid checkpoint state from the "checkpoint" file -" saver.restore () - Is used to store and restore the models -" sess.run () - Outputs the tensors and metadata obtained from running a session
Updating WAN parameters: