One Vs. Rest Logistic Regression

On their own, logistic regressions are only binary classifiers, meaning they cannot handle target vectors with more than two classes. However, there are clever extensions to logistic regression to do just that. In one-vs-rest logistic regression (OVR) a separate model is trained for each class predicted whether an observation is that class or not (thus making it a binary classification problem). It assumes that each classification problem (e.g. class 0 or not) is independent.


# Load libraries
from sklearn.linear_model import LogisticRegression
from sklearn import datasets
from sklearn.preprocessing import StandardScaler

Load Iris Flower Data

# Load data
iris = datasets.load_iris()
X =
y =

Standardize Features

# Standarize features
scaler = StandardScaler()
X_std = scaler.fit_transform(X)

Create One-Vs-Rest Logistic Regression

# Create one-vs-rest logistic regression object
clf = LogisticRegression(random_state=0, multi_class='ovr')

Train One-Vs-Rest Logistic Regression

# Train model
model =, y)

Create Previously Unseen Observation

# Create new observation
new_observation = [[.5, .5, .5, .5]]

Predict Observation’s Class

# Predict class

View Probability Observation Is Each Class

# View predicted probabilities
array([[ 0.0829087 ,  0.29697265,  0.62011865]])