v Decision Tree regression - Machine Learning

Decision Tree regression

Preliminaries

# Load libraries
from sklearn.tree import DecisionTreeRegressor
from sklearn import datasets

Load Boston Housing Dataset

# Load data with only two features
boston = datasets.load_boston()
X = boston.data[:,0:2]
y = boston.target

Create Decision Tree

Decision tree regression works similar to decision tree classification, however instead of reducing Gini impurity or entropy, potential splits are measured on how much they reduce the mean squared error (MSE):

$$\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$$

where \(y_i\) is the true value of the target and \(\hat{y}_i\) is the predicted value.

# Create decision tree classifer object
regr = DecisionTreeRegressor(random_state=0)

Train Model

# Train model
model = regr.fit(X, y)

Create Observation To Predict

# Make new observation
observation = [[0.02, 16]]

# Predict observation's value  
model.predict(observation)
array([ 33.])