Change language

Python | Tensorflow nn.softplus ()

The tensorflow.nn module provides support for many basic neural network operations.

Activation function — it is a function that is applied to the output of the neural network layer, which is then passed as input to the next layer. Activation functions are an integral part of neural networks because they provide non-linearity, without which a neural network can be reduced to a simple logistic regression model. One of the many activation functions is the Softplus function, which is defined as ,

Traditional activation functions such as sigmoidal and hyperbolic tangent, have lower and upper bounds, whereas the softplus function outputs in the range (0, ∞). The derivative of the softplus function turns out to be , which is a sigmoid function. The softplus function is very similar to the rectified liner unit (ReLU) function, the main difference being the differentiability of the softplus function at x = 0. Research paper “Improving deep neural networks using softplus blocks” by Zheng et al. (2015) suggests that softplus provides more stabilization and performance for deep neural networks than ReLU. However, ReLU is generally preferred due to its ease of computation and its derivative. Evaluating the activation function and its derivative is a common operation in neural networks, and ReLU provides faster forward and backward propagation than softplus.

nn.softplus () [alias math.softplus ] provides softplus support in Tensorflow.

Syntax : tf.nn.softplus (features, name = None) or tf .math.softplus (features, name = None)

Parameters :
features : A tensor of any of the following types: float32 , float64, int32, uint8, int16, int8, int64, bfloat16, uint16, half, uint32, uint64.
name (optional): The name for the operation.

Return type : A tensor with the same type as that of features.

Code # 1:

# Import Tensorflow library

import tensorflow as tf

 
# Constant vector of size 6

a = tf.constant ([ 1.0 , - 0.5 , 3.4 , - 2.1 , 0.0 , - 6.5 ], dtype = tf.float32)

 
# Using softplus and
# save the result in & # 39; b & # 39;

b = tf.nn.softplus (a, name = ’softplus’ )

  
# Initiate a Tensorflow session
with tf.Session () as sess:

print ( ’Input type:’ , a)

print ( ’Input:’ , sess.run (a))

print ( ’Return type:’ , b)

  print ( ’ Output: ’ , sess.run (b))

Output:

 Input type: Tensor ("Const: 0", shape = (6,), dtype = float32) Input: [1. -0.5 3.4000001 - 2.0999999 0. -6.5] Return type: Tensor ("softplus: 0", shape = (6,), dtype = float32) Output: [1.31326163e + 00 4.74076986e-01 3.43282866e + 00 1.15519524e-01 6.93147182e- 01 1.50233845e-03] 

Code # 2: Visualization

# Library import Tensorflow

import tensorflow as tf

 
# Importing the NumPy library

import numpy as np

 
# Import the matplotlib.pylot function

import matplotlib.pyplot as plt

  
# Vector size 15 with values ​​from -5 to 5

a = np.linspace ( - 5 , 5 , 15 )

 
# Using the softplus function and
# saving the result to & # 39; b & # 39;

b = tf.nn.softplus (a, name = ’softplus’ )

  
# Initiating a Tensorflow session
with tf.Session () as sess:

print ( ’Input:’ , a)

print ( ’Output:’ , sess.run (b))

plt.plot (a, sess.run (b), color = ’red’ , marker = "o"

plt.title ( "te nsorflow.nn.softplus "

  plt.xlabel ( "X"

plt.ylabel ( " Y "

 

plt .show ()

Output:

 Input: [-5. -4.28571429 -3.57142857 -2.85714286 -2.14285714 -1.42857143 -0.71428571 0. 0.71428571 1.42857143 2.14285714 2.85714286 3.57142857 4.28571429 5.] Output: [0.00671535 0.01366993 0.02772767 0.05584391 0.11093221 0.21482992 0.39846846 0.69314718 1.11275418 1.64340135 2.25378936 2.91298677 3.59915624 4.29938421 5.00671535] 

< figure class = aligncenter amp-wp-inline-e407ac51e00eb7ad9e758d070160c9d8>

Shop

Gifts for programmers

Learn programming in R: courses

$FREE
Gifts for programmers

Best Python online courses for 2022

$FREE
Gifts for programmers

Best laptop for Fortnite

$399+
Gifts for programmers

Best laptop for Excel

$
Gifts for programmers

Best laptop for Solidworks

$399+
Gifts for programmers

Best laptop for Roblox

$399+
Gifts for programmers

Best computer for crypto mining

$499+
Gifts for programmers

Best laptop for Sims 4

$

Latest questions

PythonStackOverflow

Common xlabel/ylabel for matplotlib subplots

1947 answers

PythonStackOverflow

Check if one list is a subset of another in Python

1173 answers

PythonStackOverflow

How to specify multiple return types using type-hints

1002 answers

PythonStackOverflow

Printing words vertically in Python

909 answers

PythonStackOverflow

Python Extract words from a given string

798 answers

PythonStackOverflow

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

606 answers

PythonStackOverflow

Python os.path.join () method

384 answers

PythonStackOverflow

Flake8: Ignore specific warning for entire file

360 answers

News


Wiki

Python | How to copy data from one Excel sheet to another

Common xlabel/ylabel for matplotlib subplots

Check if one list is a subset of another in Python

How to specify multiple return types using type-hints

Printing words vertically in Python

Python Extract words from a given string

Cyclic redundancy check in Python

Finding mean, median, mode in Python without libraries

Python add suffix / add prefix to strings in a list

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

Python - Move item to the end of the list

Python - Print list vertically