Logistic Regression

Binary Classification.

Logistic Regression is the simplest supervised binary classification model. Binary classification implies that our target variable is dichotomous i.e. it can take two values like 0 or 1.

The Sigmoid function:

The sigmoid function is a differentiable function heavily used in optimization problem. From the graph of logit function it is clear that function value does not vary much for large inputs (less influenced by outliers). This function is used to assign probability values in logistic regression model.

Sigmoid function is also used as an activation function in deep learning models.

Binary Cross Entropy:

Source — Google

Output size is the total number of data points used in the training.

is the ground truth of the data point

is the output of the model.

Binary Cross Entropy is used as loss function in Binary Logistic Regression.

Logistic Regression Model:

G(X) is the logistic function, θ is the weight matrix and X is the variable matrix. The output of the function is in range 0 and 1 (sigmoid output). If the value of function is greater or equal to 0.5 then the input is classified as 1 and 0 otherwise.

Python example code for logistic regression :

import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score, roc_auc_score, classification_report
from sklearn.decomposition import PCA
import pandas as pd
from utils.utils_scores import scores

flag_pca = False

data, target = load_breast_cancer(return_X_y=True)
data = pd.DataFrame(data)
target = pd.DataFrame(target)

if flag_pca:
    pca = PCA()
    data = pca.fit_transform(data)

data_train, data_test, target_train, target_test = train_test_split(data, target, test_size=0.1, random_state=0)

LogisticRegressionObject = LogisticRegression()
LogisticRegressionObject.fit(data_train, target_train)

target_test_predict = LogisticRegressionObject.predict(data_test)
scores(target_test, target_test_predict)

Example code can also be downloaded from github: https://github.com/Shubham-Saket/binary_classification_example_codes/blob/master/Logistic_Classification.py

Note: by switching from sigmoid to softmax function logistic regression can be applied on targets having more than 2 categorical values.

I hope the post helped you in understanding the basics of logistic regression.

That’s all.