A Case to Study Supervised Learning : Logistic Regression , Naive Bayes , KNN , SVM

A dataset showcasing the detailed variable analysis whether the output is dependent or independent of the given variables of the dataset.

Importing packages that are required and calling the data file on which supervised learning will be implemented.

import numpy as np
 import pandas as pd
 import matplotlib.pyplot as plt
 import seaborn as sns
 
 data=pd.read_csv(r"E:/D _ FOLDER TRANSFER/MACHINE LEARNING AI/Churn_Modelling.csv")
 
 data.isnull().sum()
 
 for value in data.columns:
  print(data[value].unique())

Data is checked for Unique values or any Null values as in "NaN" , "?" , etc then is replaced by mean or median or the most common value of that dataset .Graphically the data are plotted to analyze which values are affecting the output and which are not hence gets dropped after the data analysis.

Data Analysis

 plt.figure(figsize=(12,10))
 sns.distplot(data.CreditScore[data.Exited==0])
 sns.distplot(data.CreditScore[data.Exited==1])
 plt.legend(['no_leaving','leaving'])
 
 sns.countplot(data.Geography,hue=data.Exited)
 
 sns.countplot(data.Gender,hue=data.Exited)
 
 plt.figure(figsize=(12,10))
 sns.distplot(data.Age[data.Exited==0])
 sns.distplot(data.Age[data.Exited==1])
 plt.legend(['no_leaving','leaving'])
 
 plt.figure(figsize=(12,10))
 sns.distplot(data.Tenure[data.Exited==0])
 sns.distplot(data.Tenure[data.Exited==1])
 plt.legend(['no_leaving','leaving'])
 
 plt.figure(figsize=(12,10))
 sns.distplot(data.Balance[data.Exited==0])
 sns.distplot(data.Balance[data.Exited==1])
 plt.legend(['no_leaving','leaving'])
 
 plt.figure(figsize=(12,10))
 sns.distplot(data.NumOfProducts[data.Exited==0])
 sns.distplot(data.NumOfProducts[data.Exited==1])
 plt.legend(['no_leaving','leaving'])
 
 plt.figure(figsize=(12,10))
 sns.distplot(data.EstimatedSalary[data.Exited==0])
 sns.distplot(data.EstimatedSalary[data.Exited==1])
 plt.legend(['no_leaving','leaving'])
 
 sns.countplot(data.HasCrCard,hue=data.Exited)
 
 sns.countplot(data.IsActiveMember,hue=data.Exited)
 
 plt.figure(figsize=(12,10))
 sns.distplot(data.RowNumber[data.Exited==0])
 sns.distplot(data.RowNumber[data.Exited==1])
 plt.legend(['no_leaving','leaving'])
 
 plt.figure(figsize=(12,10))
 sns.distplot(data.CustomerId[data.Exited==0])
 sns.distplot(data.CustomerId[data.Exited==1])
 plt.legend(['no_leaving','leaving'])

Here as we can see "Age" is affecting the output hence its to be included in the training dataset.

Here as we can see "Tenure" is not affecting the output hence it's to be excluded from the training dataset.

As we can see "CreditScore" is not affecting the output that much it's to be excluded from the training dataset.

Here as we can see "Geography" is affecting the output hence its to be included in the training dataset.

Here as we can see "Gender" is affecting the output hence its to be included in the training dataset.

Here as we can see "Balance" is affecting the output hence its to be included in the training dataset.

Here as we can see "EstimatedSalary" is affecting the output hence its to be included in the training dataset.

Here as we can see "HasCrCard" is affecting the output hence its to be included in the training dataset.

Here as we can see "NumOfProducts" is affecting the output hence its to be included in the training dataset.

As we can see "RowNumber" is not affecting the output that much it's to be excluded from the training dataset.

Here as we can see "IsActiveMember" is affecting the output hence its to be included in the training dataset.

As we can see "CustomerId" is not affecting the output that much it's to be excluded from the training dataset.

data.dtypes
 from sklearn.preprocessing import LabelEncoder
 le1=LabelEncoder()
 data.Geography=le1.fit_transform(data.Geography)
 
 le2=LabelEncoder()
 data.Gender=le2.fit_transform(data.Gender)
 
 plt.figure(figsize=(12,10))
 cor=data.corr()
 sns.heatmap(cor,annot=True,cmap='coolwarm')

For ease of training all the datatypes should be encoded with either LabelEncoder() or OneHotEncoder() .

From this HeatMap its almost clear to us on what variables the output is dependent.

Dropping the data columns of which the output is independent. Hence defining Input (ip) and Output(op) .

data1=data.drop(['RowNumber','CustomerId','Surname','CreditScore',
 'Tenure'],axis=1)
 ip=data1.drop(['Exited'],axis=1)
 op=data1['Exited']
 from sklearn.preprocessing import OneHotEncoder
 from sklearn.compose import ColumnTransformer
 ct=ColumnTransformer([('geography',OneHotEncoder(),[0])],
  remainder='passthrough')
 ip=np.array(ct.fit_transform(ip),dtype=np.str)
 

Splitting the training and testing dataset by defining a test size. Using StandardScalerTranform the model is fitted. We will get different model accuracy and recall for different algorithms.

from sklearn.model_selection import train_test_split
 xtr,xts,ytr,yts=train_test_split(ip,op,test_size=0.1)
 
 from sklearn.preprocessing import StandardScaler
 sc=StandardScaler()
 sc.fit(xtr)
 sc.fit(xts)
 xtr=sc.transform(xtr)
 xts=sc.transform(xts)
 

Applying Logistic Regression

from sklearn.linear_model import LogisticRegression
 alg=LogisticRegression()
 alg.fit(xtr,ytr)
 
 yp=alg.predict(xts)
 sc.fit(x)
 x=sc.transform(x)
 alg.predict(x)
 from sklearn.metrics import accuracy_score
 
 from sklearn.metrics import recall_score
 a=accuracy_score(yts,yp)
 b=recall_score(yts,yp)
 print(a,b)
 

a=0.815
 b=0.22164948453608246

Applying Naive Bayes Classifier

from sklearn.naive_bayes import GaussianNB
 clf=GaussianNB()
 clf.fit(xtr,ytr)
 
 yp1=clf.predict(xts)
 
 from sklearn.metrics import accuracy_score
 from sklearn.metrics import recall_score
 a1=accuracy_score(yts,yp1)
 b1=recall_score(yts,yp1)
 print(a1,b1)

a1=0.815
 b1=0.3556701030927835

Applying KNN

With KNN algorithm we can tune the training by tuning the 'n_neighbors '.

from sklearn.neighbors import KNeighborsClassifier
 alg=KNeighborsClassifier(n_neighbors=3)
 alg.fit(xtr,ytr)
 
 yp2=alg.predict(xts)
 
 from sklearn.metrics import accuracy_score
 from sklearn.metrics import recall_score
 a2=accuracy_score(yts,yp2)
 b2=recall_score(yts,yp2)
 print(a2,b2)

a2=0.829
 b2=0.4845360824742268

Applying SVM

With SVM we can tune the training by tuning the 'gamma '.

from sklearn import svm
 alg1=svm.SVC(kernel='rbf',C=500,gamma=0.01)
 alg1.fit(xtr,ytr)
 yp3=alg1.predict(xts)
 
 from sklearn.metrics import accuracy_score
 from sklearn.metrics import recall_score
 a3=accuracy_score(yts,yp3)
 b3=recall_score(yts,yp3)
 print(a3,b3)
 

a3=0.861
 b3=0.4381443298969072

I tried to showcase few algorithms of Supervised Learning using same dataset as a case study.