# Weighted K-NN

Intuition:
Consider the following training set

Red marks represent 0 grade and green marks 1 grade.
Treat the white point as a query point (the point for which the class label should be predicted)

If we pass the above dataset to the kNN-based classifier, then the classifier will declare the query point to be of class 0. But it is clear from the graph that the point is closer to the points of class 1 compared to with the class. 0 points To overcome this disadvantage, a weighted kNN was used. In a weighted kNN, the nearest k points are weighted using a function called a kernel function. The intuition behind the weighted kNN is to give more weight to points that are nearby and less weight to points that are farther away. Any function can be used as a kernel function for the weight classifier knn, the value of which decreases with increasing distance. The simple function that is used is the inverse function of distance.

Algorithm :

• Let L = {(x i , y i ), i = 1 ,. ,,, n} be the training set of observations x i with the given class y i, and let x be the new observation (query point) whose class label y is to be predicted.
• Calculate d (x i , x) for i = 1 ,. ,,, n — distance between the query point and any other point in the training set.
• Select D & # 39; ⊆ D, the set of k closest training data points to the query points
• Predict the class of the query point, using distance-weighted voting. V represents class labels. Use the following formula

Implementation:
Consider 0 as a label for class 0 and 1 as a label for class 1. Below is the implementation of the weighted-kNN algorithm.

C / C++

 ` // C++ program for implementation ` ` // weighted K nearest neighbor algorithm. ` ` # include "bits / stdC++. h" ` ` using ` ` namespace ` ` std; `   ` struct ` ` Point ` ` { ` ` ` ` int ` ` val; ` ` // Dot class ` ` double ` ` x, y; ` ` // Point coordinate ` ` double ` ` distance; ` ` // Distance from reference point ` `}; `   ` // Used to sort the array of points by increasing ` ` // order of weighted distance ` ` bool ` ` comparison (Point a, Point b) ` ` { ` ` ` ` return ` ` (a.distance "b.distance); ` `} `   ` // This function finds the classification of point p using ` ` // weighted k nearest neighbor algorithm. Assumes only ` ` // two groups and returns 0 if p belongs to class 0, otherwise ` ` / / 1 (belongs to class 1). ` ` int ` ` weightedkNN (Point arr [], ` ` int ` ` n, ` ` int ` ` k, Point p) ` ` { ` ` ` ` // Fill in the weighted distances of all points from p ` ` for ` ` (` ` int ` ` i = 0; i "n; i ++) ` ` arr [i] .distance = ` ` (` ` sqrt ` ((arr [i] .x - px) * (arr [i] .x - px) +  ` (arr [i] .y - py) * (arr [i] .y - py))); `   ` // Sort points by distance weighted from p ` ` ` ` sort (arr, arr + n , comparison); `   ` // Now let’s look at the first k elements and only ` ` ` ` // two groups ` ` double ` ` freq1 = 0; ` ` // weighted sum of group 0 ` ` double ` ` freq2 = 0; ` ` // group 1 weighted sum ` ` for ` ` (` ` int ` ` i = 0; i "k; i ++) ` ` { ` ` if ` ` (arr [i] .val == 0) ` ` freq1 + = ` ` double ` ` (1 / arr [i] .distance); ` ` else ` ` if ` ` (arr [i] .val == 1) ` ` freq2 + = ` ` double ` ` (1 / arr [i] .distance); ` `} ` ` return ` ` (freq1" freq2? 0: 1); ` `} `   ` // Driver code ` ` int ` ` main () ` ` { ` ` int ` ` n = 13; ` ` // Number of data points ` ` Point arr [n]; `   ` arr  .x = 0; ` ` arr  .y = 4; ` ` arr  .val = 0; `   ` arr  .x = 1; ` ` arr  .y = 4.9; ` ` arr  .val = 0; `   ` arr  .x = 1.6; ` ` arr  .y = 5.4; ` ` arr  .val = 0; `   ` arr  .x = 2.2; ` ` arr  .y = 6; ` ` arr  .val = 0; `   ` arr  .x = 2.8; ` ` arr  .y = 7; ` ` arr  .val = 0; `   ` arr  .x = 3.2; ` ` arr  .y = 8; ` ` arr  .val = 0; `   ` arr  .x = 3.4; ` ` arr  .y = 9; ` ` arr  .val = 0; `   ` arr  .x = 1.8; ` ` arr  .y = 1; ` ` arr  .val = 1; `   ` arr  .x = 2.2; ` ` arr  .y = 3; ` ` arr  .val = 1; `   ` arr  .x = 3; ` ` arr  .y = 4; ` ` arr  .val = 1; `   ` arr  .x = 4; ` ` arr  .y = 4.5; ` ` arr  .val = 1; `   ` arr  .x = 5; ` ` arr  .y = 5; ` ` arr  .val = 1; `   ` arr  .x = 6; ` ` arr  .y = 5.5; ` ` arr  .val = 1; `   ` / * Test point * / ` ` ` ` Point p; ` ` px = 2; ` ` py = 4; `   ` // Parameter for defining the class of the query point ` ` ` ` int ` ` k = 5; ` ` printf ` ` (` ` "The value classified to query point" ` ` " is:% d. "` `, weightedkNN (arr, n, k, p)); ` ` return ` ` 0; ` `} `

python3

 ` # Python3 program to implement ` ` # weighted K nearest neighbor algorithm. `   ` import ` ` math `   ` def ` ` weightedkNN (points, p, k ` ` = ` ` 3 ` `): ` ` "" "` ` This function finds the classification of p using ` ` weighted k nearest neighbor algorithm. Only two ` ` two classes are assumed and returns 0 if p belongs to class 0, otherwise ` ` 1 (belongs to class 1). `   ` Options - ` ` glasses: Training glasses dictionary with two keys - 0 and 1 ` ` Each key has a list of training data points belonging to this `   ` p: tuple, test point of form data (x, y) `   ` k: amount nearest sos food, default 3 ` ` "" "`   ` distance ` ` = ` ` [] ` ` for ` ` group ` ` in ` ` points: ` ` for ` ` feature ` ` in ` ` points [group]: `   ` # calculate Euclidean distance p from training points ` ` euclidean_distance ` ` = ` ` math. sqrt ((featu re [` ` 0 ` `] ` ` - ` ` p [` ` 0 ` `]) ` ` * ` ` * ` ` 2 ` ` + ` ` (feature [` ` 1 ` `] ` ` - ` ` p [` ` 1 ` `]) ` ` * ` ` * ` ` 2 ` `) ` ` `  ` # Add the form tuple (distance, group) to the distance list ` ` distance.append ((euclidean_distance, group)) `   ` ` ` # sort the distance list in ascending order ` ` ` ` # and select the first k distances ` ` distance ` ` = ` ` sorted ` ` (distance) [: k] `   ` freq1 ` ` = ` ` 0 ` ` # group 0 weighted sum ` ` freq2 ` ` = ` ` 0 ` ` # group 1 weighted sum `   ` ` ` for ` ` d ` ` in ` ` distance: ` ` if ` ` d [` ` 1 ` `] ` ` = ` ` = ` ` 0 ` `: ` ` freq1 ` ` + ` ` = ` ` (` ` 1 ` ` / ` ` d [` ` 0 ` `]) `   ` elif ` ` d [` ` 1 ` `] ` ` = ` ` = ` ` 1 ` `: ` ` freq2 ` ` + ` ` = ` ` (` ` 1 ` ` / ` ` d [` ` 0 ` `]) `     ` return ` ` 0 ` ` if ` ` freq1" freq2 ` ` else ` ` 1 `   ` # Driver function ` ` def ` ` main (): ` ` `  ` ` ` # Training point dictionary with two keys - 0 and 1 ` ` # key 0 has points belonging to class 0 ` ` # key 1 has class 1 points `   ` ` ` points ` ` = ` ` {` ` 0 ` `: [(` ` 0 ` `, ` ` 4 ` `), (` ` 1 ` `, ` ` 4.9 ` `), (` ` 1.6 ` `, ` ` 5.4 ` `), (` ` 2.2 ` `, ` ` 6 ` `), (` ` 2.8 ` `, ` ` 7 ` `), (` ` 3.2 ` `, ` ` 8 ` `), (` ` 3.4 ` `, ` ` 9 ` `)], ` ` 1 ` `: [(` ` 1.8 ` `, ` ` 1 ` `), (` ` 2.2 ` `, ` ` 3 ` `), (` ` 3 ` `, ` ` 4 ` `), (` ` 4 ` `, ` ` 4.5 ` `), (` ` 5 ` `, ` ` 5 ` `), (` ` 6 ` `, ` ` 5.5 ` `)]} `   ` # query point p (x, y) ` ` p ` ` = ` ` (` ` 2 ` `, ` ` 4 ` `) `   ` Number of neighbors ` ` k ` ` = ` ` 5 ` ` `  ` print ` ` (` ` "The value classified to query point is: {} "` `. ` ` format ` ` (weightedkNN (points, p, k))) `   ` if ` ` __ name__ ` ` = ` ` = ` ` ’__main__’ ` `: ` ` main () `

Exit:

` The value classified to query point is: 1 `

Time complexity : O (N), where N — the number of points in the training set.

## Shop Learn programming in R: courses

\$ Best Python online courses for 2022

\$ Best laptop for Fortnite

\$ Best laptop for Excel

\$ Best laptop for Solidworks

\$ Best laptop for Roblox

\$ Best computer for crypto mining

\$ Best laptop for Sims 4

\$

Latest questions

NUMPYNUMPY

psycopg2: insert multiple rows with one query

NUMPYNUMPY

How to convert Nonetype to int or string?

NUMPYNUMPY

How to specify multiple return types using type-hints

NUMPYNUMPY

Javascript Error: IPython is not defined in JupyterLab

## Wiki

Python OpenCV | cv2.putText () method

numpy.arctan2 () in Python

Python | os.path.realpath () method

Python OpenCV | cv2.circle () method

Python OpenCV cv2.cvtColor () method

Python - Move item to the end of the list

time.perf_counter () function in Python

Check if one list is a subset of another in Python

Python os.path.join () method