Demystifying the Confusion Matrix: A Deep Dive into Evaluating Model Performance
In the world of Machine Learning, understanding model performance is essential. One powerful tool for this purpose is the Confusion Matrix, a simple yet highly effective table layout for visualization and comprehension of your classifier’s performance.
The confusion matrix places the model’s predictions against the ground-truth data labels, creating an intuitive comparison grid. Each cell in this matrix represents a different aspect of the model’s performance, namely True Negatives (TN), False Negatives (FN), False Positives (FP) and True Positives (TP).
Understanding the Confusion Matrix
True Negatives (TN)
The top-left cell in the matrix represents True Negatives. This indicates instances where the model predicted a negative result, and the actual label was indeed negative. In other words, these are the instances where the model correctly predicted a negative outcome.
False Negatives (FN)
The top-right cell represents False Negatives, indicating times when the model predicted a negative outcome, but the actual label was positive. This represents instances where the model should have predicted a positive outcome but did not.
False Positives (FP)
The bottom-left cell signifies False Positives. These are instances where the model predicted a positive outcome, but the actual label was negative. This denotes the occurrences where the model incorrectly predicted a positive outcome.
True Positives (TP)
Lastly, the bottom-right cell stands for True Positives, indicating times when the model correctly predicted a positive outcome.
To summarize, the key components of the confusion matrix are:
- TP = True positives: a positive label is correctly predicted.
- TN = True negatives: a negative label is correctly predicted.
- FP = False positives: a negative label is predicted as a positive.
- FN = False negatives: a positive label is predicted as a negative.
Key Performance Metrics Derived from the Confusion Matrix
Accuracy
Accuracy is calculated as the number of correct predictions divided by the total number of predictions. It is a quick indicator of the overall performance of the model. However, accuracy may not be reliable when dealing with imbalanced datasets.
More generec accuracy fomularization:
accuracy = number of correct predictions/total number of predictions
Accuracy from pobability point of view:
accuracy = (TP + TN) / number of samples
Sensitivity/Recall (True Positive Rate)
Sensitivity or Recall is a measure of the proportion of actual positive cases that were correctly identified by the model.
recall = sensitivity = TP / (TP + FN)
Specificity
Specificity denotes the fraction of actual negative cases that were correctly identified by the model.
specificity = TN / (TN + FP)
Precision
Precision represents the proportion of predicted positive cases that were correctly identified.
precision = TP / ( TP + FP)
False Positive Rate (FPR)
False Positive Rate denotes how often negative instances are incorrectly identified as positive.
false_positive_rate = FP / ( FP + TN )
Key Note: The sum of specificity and false positive rate should always be 1!
True Positive Rate (sensitivity)
This metric illustrates how often “true” labels are correctly identified as “True”.
False Positive Rate (False Alarm Rate)
This illustrates how often “False” labels are incorrectly identified as “True”.
Wrapping Up
The confusion matrix and its derived metrics provide a powerful and intuitive framework to evaluate the performance of classification models. By understanding each element and the relationships between them, we can gain valuable insights into our model’s strengths and weaknesses, and take steps to improve its performance.