HomeArtificial IntelligenceMachine Learning

# Confusion matrices

Accuracy alone can be misleading. For example, if most observations are labelled 1, predicting 1 every time can give a high accuracy score without fitting a model.

  • Accuracy alone hides the distribution of predictions across classes.
  • It also hides which kinds of mistakes the model makes.

A confusion matrix, also called an error matrix, compares predicted and actual classes. In binary classification, it records four counts:

  • True positives (TP): actual positives predicted as positive.
  • True negatives (TN): actual negatives predicted as negative.
  • False positives (FP): actual negatives predicted as positive.
  • False negatives (FN): actual positives predicted as negative.

Other metrics you can derive from those counts

Metric Formula What it measures
Accuracy TP+TNTP+TN+FP+FN\frac{TP+TN}{TP+TN+FP+FN} Fraction of all predictions that are correct
Misclassification rate FP+FNTP+TN+FP+FN\frac{FP+FN}{TP+TN+FP+FN} Fraction of all predictions that are wrong
Recall TPTP+FN\frac{TP}{TP+FN} Fraction of actual positives correctly identified
Specificity TNTN+FP\frac{TN}{TN+FP} Fraction of actual negatives correctly identified
Precision TPTP+FP\frac{TP}{TP+FP} Fraction of positive predictions that are correct
F1 score 2×Precision×RecallPrecision+Recall\frac{2\times Precision\times Recall}{Precision+Recall} Harmonic mean of precision and recall