Measuring results in Classifiers with Precision and Recall

I was trying to explain precision & recall to a few colleagues and realized that the same explanation does not work for everyone. After making this profound statement, imagine me holding up a 'Super Genius' sign (remember  Wile E. Coyote ?)

A bit of context — Precision and recall are two key numbers to evaluate a classification algorithm. Classification algorithms are a category of machine  learning algorithms that answer the question -- What bucket does the item under consideration belong? There are several types of classification algorithms. You can read details on them at: binary, multi-label and multi-class or check out the image below.

For classifiers, you get to choose which predictions are the most important to you. In the case of binary classifiers, there are four buckets the predictions can fall into.

• True Positive : Predicted positive and is correct
• False Positive : Predicted positive and is incorrect
• True Negative : Predicted negative and is correct
• False Negative : Predicted negative and is incorrect

I'll walk through two examples to show when you might be interested in positive and negative results.

Example 1: My classifier chooses videos that are kid friendly. In this case, I'd like to have high confidence if the classifier predicts a video as kid friendly, it is. I am willing to accept the tradeoff that some videos will be marked as kid unfriendly when they should not be. This translates to improving the True Positives (TP) or high precision.

Example 2: My classifier scans images images looking for a match for a person's face. In this case, I care the classifier does not miss a potential match and I'm willing to pay the price in scanning through additional faces. This translates to improving True Negatives (TN) or high recall.

There are additional items to consider and one of the key visualizations is a confusion matrix. The link for the confusion matrix also provides a reasonable overview for the visualization as well as a few other items that can be used to measure the effectiveness of classifiers.

To wrap up, higher precision indicates higher trust in predicted positives, while higher recall indicates higher trust in predicted negatives.