Algorithms drive the Machine Learning world

Among them, there’s an algorithm often labeled as lazy. But it’s quite a performer when it comes to classifying data points. It’s called the k-nearest neighbours algorithm and is often quoted as one of the most important machine learning algorithms.

  • The k-nearest neighbours (KNN) algorithm is a data classification method for estimating the likelihood that a data point will become a member of one group or another based on what group the data points nearest to it belong to.
  • The k-nearest neighbour algorithm is a type of Supervised learning algorithm used to solve classification and regression problems. However, it’s mainly used for classification problems.
  • K-NN is a non-parametric algorithm, which means it does not make any assumption on underlying data.
  • It is also called a lazy learner algorithm because it does not learn from the training set immediately instead it stores the dataset and at the time of classification, it performs an action on the dataset.
  • KNN algorithm at the training phase just stores the dataset and when it gets new data, then it classifies that data into a category that is much similar to the new data.
  • Example: Suppose, we have an image of a creature that looks similar to cat and dog, but we want to know either it is a cat or dog. So for this identification, we can use the KNN algorithm, as it works on a similarity measure. Our KNN model will find the similar features of the new data set to the cats and dogs images and based on the most similar features it will put it in either cat or dog category.

Why do we need KNN

Suppose there are two categories, i.e., Category A and Category B, and we have a new data point x1, so this data point will lie in which of these categories. To solve this type of problem, we need a K-NN algorithm. With the help of K-NN, we can easily identify the category or class of a particular dataset. Consider the below diagram:

The K-NN working can be explained on the basis of the below algorithm:

  • Step-1: Select the number K of the neighbours
  • Step-2: Calculate the Euclidean distance of K number of neighbours
  • Step-3: Take the K nearest neighbours as per the calculated Euclidean distance.
  • Step-4: Among these k neighbours, count the number of the data points in each category.
  • Step-5: Assign the new data points to that category for which the number of the neighbour is maximum.
  • Step-6: Our model is ready.

How to select the value of K in the K-NN Algorithm?

Below are some points to remember while selecting the value of K in the K-NN algorithm:

  • There is no particular way to determine the best value for “K”, so we need to try some values to find the best out of them. The most preferred value for K is 5.
  • A very low value for K such as K=1 or K=2, can be noisy and lead to the effects of outliers in the model.
  • Large values for K are good, but it may find some difficulties.

Advantages of KNN Algorithm:

  • It is simple to implement.
  • It is robust to the noisy training data
  • It can be more effective if the training data is large.

Disadvantages of KNN Algorithm:

  • Always needs to determine the value of K which may be complex some time.
  • The computation cost is high because of calculating the distance between the data points for all the training samples.

Python Implementation of KNN

  1. To implement the KNN algorithm we need to install the Scikit-Learn package in our system by using the following command pip install -U scikit-learn
  2. Import iris Dataset from the package
  3. Create feature and target variables
  4. Split the Data into training and testing Data
  5. Generate a KNN Model using neighbours value
  6. Train or fit the data into the model
  7. predict the future

Source Code

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris
iris = load_iris()
df=pd.DataFrame(iris.data,columns=iris.feature_names)
df['target'] =iris.target
x=df.drop(['target'],axis='columns')
y=df.target
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=1)
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(x_train,y_train)
print(knn.score())