Python | CAP — cumulative accuracy profile analysis

Michael Zippo 18.07.2021

The CAP, commonly referred to as the Cumulative Accuracy Profile, is used to evaluate the performance of a classification model. This helps us understand and conclude about the robustness of the classification model. To visualize this, our graph depicts three different curves:

Random plot
Plot generated using SVM classifier or random forest classifier
Ideal plot (ideal line)

We are working with data to understand concept.

Code: load dataset.

# import libraries

import pandas as pd

import seaborn as sns

import matplotlib. pyplot as plt

import numpy as np

# loading dataset

data = pd.read_csv ( ’C: Users DELL Desktop Social_Network_Ads .csv’ )

print ( "Data Head:" , data .head ())

Output:

 Data Head: User ID Gender Age Estimated Salary Purchased 0 15624510 Male 19 19000 0 1 15810944 Male 35 20000 0 2 15668575 Female 26 43000 0 3 15603246 Female 27 57000 0 4 15804002 Male 19 76000 0

Code: data input / output.

# Input and output

x = data.iloc [:, 2 : 4 ]

y = data.iloc [:, 4 ]

print ( "Input:" , x.iloc [ 0 : 10 ,:])

Exit:

 Input: Age EstimatedSalary 0 19 19000 1 35 20000 2 26 43000 3 27 57000 4 19 76000 5 27 58000 6 27 84000 7 32 150000 8 25 33000 9 35 65000

Code: Split dataset for training and testing.

# data splitting

from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split (

x, y, test_size = 0.3 , random_state = 0 )

Code: random forest classifier

# classifier

from sklearn.ensemble import RandomForestClassifier

classifier = RandomForestClassifier (n_estimators = 400 )

# advanced training
classifier.fit (x_train, y_train)

#predict

pred = classifier.predict (x_test)

Code: Finding the accuracy classifier.

# Performance Model

from sklearn.metrics import accuracy_score

print ( " Accuracy : " , accuracy_score (y_test, pred) * 100 )

Output:

 Accuracy: 91.66666666666666

Random model

the number of points in the range from 0 to the total number of data points in the dataset. The y-axis was stored as the total number of points for which the dependent variable from our dataset has a result of 1. A random plot can be understood as a linearly increasing relationship. An example is a model that predicts whether a product was purchased (positive result) by each person in a group of people (classifying parameter) based on factors such as their gender, age, income, etc. If group members are contacted randomly, the total number of products sold will grow linearly to a maximum value corresponding to the total number of buyers in the group. This distribution is called a "random" CAP .

Code: random model

# code for random plot

import matplotlib.pyplot as plt

import numpy as np

# test data length

total = len (y_test)

# Counting 1 marks in test data

one_count = np. sum (y_test)

# counting tags & # 39; 0 & # 39; in test data

zero_count = total - one_count

plt.figure (figsize = ( 10 , 6 ))

# x-axis ranges from 0 to the total number of people contacted
# Y-axis ranges from 0 to total positives.

plt.plot ([ 0 , total], [ 0 , one_count], c = ’b’ ,