v Logistic Regression - Machine Learning

Logistic Regression

Despite having "regression" in its name, a logistic regression is actually a widely used binary classifier (i.e. the target vector can only take two values). In a logistic regression, a linear model (e.g. \(\beta_{0}+\beta_{1}x\)) is included in a logistic (also called sigmoid) function, \({\frac {1}{1+e^{-z}}}\), such that:

$$P(y_i=1 \mid X)={\frac {1}{1+e^{-(\beta_{0}+\beta_{1}x)}}}$$

where \(P(y_i=1 \mid X)\) is the probability of the \(i\)th observation's target value, \(y_i\), being class 1, \(X\) is the training data, \(\beta_0\) and \(\beta_1\) are the parameters to be learned, and \(e\) is Euler's number.

Preliminaries

# Load libraries
from sklearn.linear_model import LogisticRegression
from sklearn import datasets
from sklearn.preprocessing import StandardScaler

Load Iris Flower Dataset

# Load data with only two classes
iris = datasets.load_iris()
X = iris.data[:100,:]
y = iris.target[:100]

Standardize Features

# Standarize features
scaler = StandardScaler()
X_std = scaler.fit_transform(X)

Create Logistic Regression

# Create logistic regression object
clf = LogisticRegression(random_state=0)

Train Logistic Regression

# Train model
model = clf.fit(X_std, y)

Create Previously Unseen Observation

# Create new observation
new_observation = [[.5, .5, .5, .5]]

Predict Class Of Observation

# Predict class
model.predict(new_observation)
array([1])

View Predicted Probabilities

# View predicted probabilities
model.predict_proba(new_observation)
array([[ 0.18823041,  0.81176959]])