Eval

The classeval library contains various functions to plot the results. The main function to plot is the function classeval.classeval.plot(). This function automatically determines whether the trained model is based on a two-class or multi-class approach. Plotting results is possible by simply using the classeval.classeval.plot() functionality.

Lets first train a simple model:

# Load data
X, y = clf.load_example('breast')
X_train, X_test, y_train, y_true = train_test_split(X, y, test_size=0.2)

# Fit model
model = gb.fit(X_train, y_train)
y_proba = model.predict_proba(X_test)[:,1]
y_pred = model.predict(X_test)

Evaluate the models performance and use the output for making the plots.

# Import library
import classeval as clf

# Evaluate model
out = clf.eval(y_true, y_proba, pos_label='malignant')

# Make plot
ax = clf.plot(out)

Two-class

ROC-AUC

Plotting the ROC is initially desiged for two-class models. However, with some adjustments it is also possible to plot the ROC for a multi-class model under the assumption that is OvR or OvO schemes. See methods section for more details. With the function classeval.ROC.eval() the ROC and AUC can be examined and plotted for both the two-class as well as multi-class models.

# Compute ROC
outRoc = clf.ROC.eval(y_true, y_proba, pos_label='malignant')

# Make plot
ax = clf.ROC.plot(outRoc)

Two-class

Multi-class

CAP

# Compute CAP
outCAP = clf.CAP(out['y_true'], out['y_proba'], showfig=True)

AP

# Compute AP
outAP = clf.AP(out['y_true'], out['y_proba'], showfig=True)

Confusion matrix

A confusion matrix is a table to describe the performance of a classification model. With the function classeval.confmatrix.eval() the confusion matrix can be examined and plotted for both the two-class as well as multi-class model.

# Compute confmatrix
outCONF = clf.confmatrix.eval(y_true, y_pred)

# Make plot
ax = clf.confmatrix.plot(outCONF)

Two-class

Multi-class

Probability Plot

The probability plot depicts the probabilities of the samples being classified. This function is desiged for only two-class models and callable via: classeval.classeval.TPFP()

# Compute TPFP
out = clf.TPFP(out['y_true'], out['y_proba'], showfig=True)

---

---