 # ML | Logistic regression using Tensorflow

Brief description of logistic regression:
Logistic regression — it is a classification algorithm commonly used in machine learning. This allows you to classify data into distinct classes by examining relationships from a given set of labeled data. It examines linear relationships from a given dataset and then introduces nonlinearity as a sigmoidal function.

In the case of logistic regression, the hypothesis is a straight line sigmoid, i.e. where Where is the vector ` w represents the weights, and the scalar b represents the offset of the model. Let`s render the sigmoid function — `

``` import numpy as np import matplotlib.pyplot as plt   def sigmoid (z): return 1 / ( 1 + np.exp ( - z))   plt.plot (np.arange ( - 5 , 5 , 0.1 ), sigmoid (np.arange ( - 5 , 5 , 0.1 ))) plt.title ( `Visualization of the Sigmoid Function` )   plt.show () ```

` `

Output: Note that the range of the Sigmoid function is (0, 1), which means that the resulting values ​​are between 0 and 1. This property of the Sigmoid function makes it a really good choice of activation function for binary classification. Also ` for z = 0, Sigmoid (z) = 0.5 ` which is the midpoint of the range of the sigmoidal function.

As with linear regression, we need to find the optimal values ​​for w and b, for which the cost function J is minimal. In this case, we will use the Sigmoid Cross Entropy cost function which is defined This cost function will then be optimized with using gradient descent.

Implementation:
Let`s start by importing the required libraries. We`ll be using Numpy along with Tensorflow for calculations, Pandas for basic data analysis, and Matplotlib for plotting. We will also use the ` Scikit-Learn ` preprocessor module for One Hot Encoding data.

 ` # module import ` ` import ` ` numpy as np ` ` import ` ` pandas as pd ` ` import ` ` tensorflow as tf ` ` import ` ` matplotlib.pyplot as plt ` ` from ` ` sklearn.preprocessing ` ` import ` ` OneHotEncoder ```` ```

Next we will import dataset . We will use a subset of the well-known Iris dataset .

 ` data ` ` = ` ` pd.read_csv (` ` `dataset. csv` ` `, header ` ` = ` ` None ` `) ` ` print ` ` (` ` "Data Shape:" ` `, data.shape) `   ` print ` ` (data.head ()) `

Exit:

` Data Shape: (100, 4) 0 1 2 3 0 0 5.1 3.5 1 1 1 4.9 3.0 1 2 2 4.7 3.2 1 3 3 4.6 3.1 1 4 4 5.0 3.6 1 `

Now let`s semi read the matrix of objects and the corresponding labels and render.

 ` # Feature Matrix ` ` x_orig ` ` = ` ` data.iloc [:, ` ` 1 ` `: ` ` - ` ` 1 ` `]. values ​​`   ` # Data Labels ` ` y_orig ` ` = ` ` data.iloc [:, ` ` - ` ` 1 ` `:]. values ​​`   ` print ` ` (` ` "Shape of Feature Matrix : "` `, x_orig.shape) ` ` print ` ` (` ` "Shape Label Vector:" ` `, y_orig.shape) `

Output :

` Shape of Feature Matrix: (100, 2) Shape Label Vector: (100, 1) `

Render the data to the data.

` `

``` Amount of positive data x_pos = np.array ([x_orig [i] for i in range ( len (x_orig)) if y_orig [i] = = 1 ])   # Negative data points x_neg = np.array ([x_orig [i] for i in range ( len (x_orig)) if y_orig [i] = = 0 ])   # Plot positive data points plt.scatter (x_pos [:, 0 ], x_pos [:, 1 ], color = ` blue` , label = `Positive` )    # Plot negative data points plt.scatter (x_neg [:, 0 ], x_neg [:, 1 ], color = `red` , label = `Negative` )    plt.xlabel ( ` Feature 1` ) plt.ylabel ( `Feature 2` ) plt.title ( `Plot of given data` ) plt.legend ()   plt.show () ``` ,

Now we will be One Hot Encoding for the data to work with the algorithm. One hot coding converts categorical features into a format that works better with classification and regression algorithms. We will also set the learning rate and the number of epochs.

 ` # Create One Hot Encoder ` ` oneHot ` ` = ` ` OneHotEncoder () `   ` # x_orig encoding ` ` oneHot.fit (x_orig) ` ` x ` ` = ` ` oneHot.transform (x_orig) .toarray () `   ` # y_orig encoding ` ` oneHot.fit (y_orig) ` ` y ` ` = ` ` oneHot.transform (y_orig) .toarray () `   ` alpha, epochs ` ` = ` ` 0.0035 ` `, ` ` 500 ` ` m, n ` ` = ` ` x.shape ` ` print ` ` (` ` `m = `` `, m) ` ` print ` ` ( ` ` `n =` ` `, n) ` ` print ` ` (` ` `Learning Rate =` ` `, alpha) ` ` print ` ` (` ` `Number of Epochs =` , epochs) ```` ```

Exit :

` m = 100 n = 7 Learning Rate = 0.0035 Number of Epochs = 500 `

Now we will start creating a model with a subdividing the ` X ` and ` Y ` placeholders so that we can feed our tutorials ` x ` and ` y ` to the optimizer as we learn. We will also create ` W ` and ` b ` trainable variables that can be optimized with the Gradient Descent Optimizer.

 ` # There are n columns in the object matrix ` ` # after one hot encoding. ` ` X ` ` = ` ` tf.placeholder (tf.float32, [` ` None ` `, n]) `   ` # Since this is a binary classification problem, ` ` # Y can only take 2 values. ` ` Y ` ` = ` ` tf.placeholder (tf.float32, [` ` None ` `, ` ` 2 ` `]) `   ` # Learning weight variables ` ` W ` ` = ` ` tf.Variable (tf.zeros ([n, ` ` 2 ` `])) `   ` # Learning variable offset ` ` b ` ` = ` ` tf.Variable (tf.zeros ([` ` 2 ` `])) `

Now declare hypothesis, cost function, optimizer and an initializer for global variables.

 ` # Hypothesis ` ` Y_hat ` ` = ` ` tf.nn.sigmoid (tf .add (tf.matmul (X, W), b)) `   ` # Sigmoid Cross Entropy Cost Function ` ` cost ` ` = ` ` tf.nn.sigmoid_cross_entropy_with_logits (` ` ` ` logits ` ` = ` ` Y_hat, labels ` ` = ` ` Y) `   ` # Gradient Descent Optimizer ` ` optimizer ` ` = ` ` tf.train.GradientDescentOptimizer (` ` learning_rate ` ` = ` ` alpha) .minimize ( cost) `   ` # Global Variables Initializer ` ` init ` ` = ` ` tf.global_variables_initializer () `

Start the tutorial process within a Tensorflow session.

 ` # Start a Tensorflow session ` ` with tf.Session () as sess: `   ` # Initializing variables ` ` sess.run (init) `   ` # Lists for storing varying value and accuracy at each epoch ` ` cost_history, accuracy_history ` ` = ` ` [], [] `   ` # Loop over all eras ` ` for ` ` epoch ` ` in ` ` range ` ` (epochs): ` ` cost_per_epoch ` ` = ` ` 0 `   ` # Launching the Optimizer ` ` sess.run (optimizer, feed_dict ` ` = ` ` {X: x, Y: y}) `   ` ` ` # Calculate cost for the current era ` ` c ` ` = ` ` sess.run (cost, feed_dict ` ` = ` ` {X: x, Y: y}) ` ` `  ` # Calculate accuracy for the current epoch ` ` correct_prediction ` ` = ` ` tf.equal (tf.argmax (Y_hat , ` ` 1 ` `), ` ` tf.argmax (Y, ` ` 1 ` `)) ` ` accuracy ` ` = ` ` tf.re duce_mean (tf.cast (correct_prediction, ` ` tf.float32)) `   ` # Save cost and accuracy in history ` ` cost_history.append (` ` sum ` ` (` ` sum ` ` (c))) ` ` accuracy_history.append (accuracy. ` ` eval ({X: x, Y: y}) * 100 ) ````      # Display the result in the current era if epoch % 100 = = 0 and epoch! = 0 : print ( "Epoch" + str (epoch) + " Cost: "   + str (cost_history [ - 1 ]))   Weight = sess.run (W) # Optimized weight Bias = sess.run (b)  # Optimized slope   # Final precision correct_prediction = tf.equal (tf.argmax (Y_hat, 1 ),   tf.argmax (Y, 1 )) accuracy = tf.reduce_mean (tf.cast (correct_prediction,  tf.float32))   print ( " Accuracy: " , accuracy_history [ - 1 ], "%" ) ```

Output:

` Epoch 100 Cost: 125.700202942 Epoch 200 Cost: 120.647117615 Epoch 300 Cost: 118.151592255 Epoch 400 Cost: 116.549999237 Accuracy: 91.0000026226% `

Let`s outline the cost change during eras.

 ` plt.plot (` ` list ` ` (` ` range ` ` (epochs)), cost_history) ` ` plt.xlabel (` ` `Epochs` ` `) ` ` plt.ylabel (` ` `Cost` ` `) ` ` plt.title (` ` `Decrease in Cost with Epochs` ` `) `   ` plt.show () ` Graph precision changes by epoch.

 ` plt.plot (` ` list ` ` (` ` range ` ` (epochs)), accuracy_history) ` ` plt.xlabel (` ` `Epochs` ` `) ` ` plt.ylabel (` ` `Accuracy` ` `) ` ` plt.title (` ` `Increase in Accuracy with Epochs` ` `) `   ` plt.show () ` We will now build a Decision Boundary for our trained classifier. Decision boundary — it is a hypersurface that divides the base vector space into two sets, one for each class.

 ` # Calculate decision boundary ` ` decision_boundary_x ` ` = ` ` np.array ([np. ` ` min ` ` (x_orig [:, ` ` 0 ` `]), ` ` np. ` ` max (x_orig [:, 0 ])]) ````   decision_boundary_y = ( - 1.0 / Weight [ 0 ]) * (decision_boundary_x * Weight + Bias)   decision_boundary_y = [ sum (decision_boundary_y [:, 0 ]),  sum (decision_boundary_y [:, 1 ])]   Amount of positive data x_pos = np.array ([x_orig [i] for i in range ( len (x_orig)) if y_orig [i] = = 1 ])   # Negative data points x_neg = np.array ([x_orig [i] for i in range ( len (x_orig))  ````  ` ` if ` ` y_orig [i] ` ` = ` ` = ` ` 0 ` `]) `   ` # Plot positive data points ` ` plt.scatter (x_pos [:, ` ` 0 ` `], x_pos [:, ` ` 1 ` `], ` ` color ` ` = ` ` `blue` ` `, label ` ` = ` `` Positive` ` `) `   ` # Plot negative data points ` ` plt.scatter (x_neg [:, ` ` 0 ` `], x_neg [:, ` ` 1 ` `], ```` color = `red` , label = ` Negative` )   # Building a decision boundary plt.plot (decision_boundary_x, decision_boundary_y) plt.xlabel ( `Feature 1` ) plt. ylabel ( `Feature 2` ) plt.title ( `Plot of Decision Boundary` ) plt.legend ()    plt.show () ``` code class = "keyword"> = ` = ` ` 0 ` `]) `

` # Plot positive data points `

` plt.scatter (x_pos [:, ` ` 0 ` `], x_pos [:, 1 ], `

``` color = `blue` , label = ```
``` ```
``` ```
``` ```
``` (adsbygoogle = window.adsbygoogle || []).push({}); © 2021 Python.Engineering Best Python tutorials books for beginners and professionals Become an author and write for us Architect Development For dummies Machine Learning Analysis Loops Counters NumPy NLP Regular Expressions File Handling Arrays String Variables Knowledge Database X Submit new EBook \$(document).ready(function () { \$(".modal_galery").owlCarousel({ items: 1, itemsCustom: false, itemsDesktop: [1300, 1], itemsDesktopSmall: [960, 1], itemsTablet: [768, 1], itemsTabletSmall: false, itemsMobile: [479, 1], singleItem: false, itemsScaleUp: false, pagination: false, navigation: true, rewindNav: true, autoPlay: true, stopOnHover: true, navigationText: [ "<img class='img_no_nav_mob' src='/wp-content/themes/nimani/image/prevCopy.png'>", "<img class='img_no_nav_mob' src='/wp-content/themes/nimani/image/nextCopy.png'>" ], }); \$(".tel_mask").mask("+9(999) 999-99-99"); }) ```