Metrics

This section describes the metrics implemented in neuro. A metrics is used to assess how well the neural network performs on a data set.

Accuracy

The accuracy is computed as the ratio of correctly labeled samples to the total number of samples:

$$\frac{\text{# correctly classified samples}}{\text{# samples}}$$

For a multiclass classification problem, the output of the network for a single sample will be of the form

$$\boldsymbol{\hat{y}} = \begin{bmatrix} \hat{y}_1 & \hat{y}_2 & \dots & \hat{y}_C \end{bmatrix}^T$$

where $\hat{y}_i$ is the probability that the sample belongs to class $i$ and $C$ the number of classes. The predicted class for that sample is thus

$$\hat{c} = \arg \max_i \hat{y}_i$$

For a binary classification problem, the output of the network will be a single number, $\hat{y}$. If the data set contains two classes, 0 and 1, then the predicted class will be:

$$\hat{c} = \begin{cases} 0 &\text{if } \hat{y} < 0.5 \newline 1 &\text{if } \hat{y} \geq 0.5 \end{cases}$$

In both multiclass and binary classifications, if $\hat{c}$ matches the true label of the sample, then the sample is counted as correctly labeled.

F-Score

Note: The F-Score is not yet implemented in the library.

In order to define the F-score, we first need to introduce the confusion matrix, the precision, P, and recall, R. The confusion matrix helps us visualize how the neural network classified the samples by looking at the samples that have been correctly classified and those that haven’t. The following image shows a confusion matrix for a binary classification problem with classes 0 and 1.

The precision is a measure of the fraction of samples classified as positive that are actually positive. It is defined as

$$P = \frac{\text{true positive}}{\text{true positive + false positive}}$$

The recall is a measure of the fraction of actual positives that were classified as positive. It expression is

$$R = \frac{\text{true positive}}{\text{true positive + false negative}}$$

The F-score is then defined as the harmonic mean of the precision and recall:

$$F_1 = \frac{2}{\frac{1}{P}+\frac{1}{R}} = \frac{2PR}{P+R}$$