Sunday, December 25, 2016

False positive vs. false negative

Detection systems are nearly ubiquitous. These include medical diagnostic tests, antivirus software, terrorist and criminal screening algorithms, and financial fraud detection systems. Although complex and sophisticated, these systems are 100% accurate.

A false positive result occurs when you detect the occurrence of something that hasn't truly occurred. (i.e., a test for influenza is positive in the absence of infection).  A false negative result occurs when you fail to detect something that is truly there. (i.e., a test for influenza is negative when I really am infected).

False positive and false negative rates are linked. Making a test more sensitive decreases the false negative rate, but increases the false positive rate.  Making a test more specific decreases the false positive rate, but increases the false negative rate. An initial screening test tends to be sensitive but is often followed by a confirmatory test which is more specific. An antibody-based test might be followed by a PCR test specific to the pathogen/disease in question. A mammogram that detects a mass might be followed by a biopsy and pathology examination. A positive airport screening detection (e.g. millimeter wave machine) is followed by a pat down.

Generally speaking, the consequences of an incorrect result drive the decision toward a more sensitive or more specific test.  Failure to correctly determine whether or not I have influenza generally has a minor consequence. Failure to correctly detect that I have HIV or that I'm carrying a weapon potentially has a much greater consequence.

If the goal is to not miss any cases of disease or weapons, there will be people incorrectly diagnosed as having influenza (or not carrying weapons) by the test or algorithm. If there is low tolerance for false negatives (i.e. I really do have the condition but the test is negative) and a sensitive detection system is used, there will be a higher false positive rate. The idea is to make sure that nobody who really is positive for the condition goes undetected. Nobody wants to miss a serious medical condition or hidden weapons. This does mean that there are people who receive medical tests and security pat downs that they don't truly "need" because they weren't truly positive.