Overdiagnosis: technical versus clinical false positive [2]

Whether or not a false positive is an error can depend on who you ask

Welcome to Limits of Inference! The post below is not intended as a self-standing piece. This is some supplementary context in support of a previous article. To get an introduction to the problem of overdiagnosis, check out the original piece here.

Questions? Concerns? Let me know in the comments. Also, subscribe below.

Defining a false-positive is often the most contentious part of a discussion on overdiagnosis. Is the metal detector there to detect metal? Or is the metal detector there to prevent violent crime? Is it a false-positive if there was metal but no weapon? What if the person had a weapon but didn’t intend to commit a crime?

These debates tend to occur in the world of ‘but, technically.’ Differentiating between a false-positive that did not mean what the patient or user understood it to mean and a true technical false positive can start fights. For example, most pregnancy test guidance says false-positive tests are virtually impossible. However, a woman can test positive when not pregnant, for example, if she were pregnant in the past and had an early miscarriage. A person wondering if there will be a baby in ten months reasonably understands this scenario as a false-positive. The scientists or engineers who designed the test will insist these are not false positives but just a different and valid explanation for a positive pregnancy test other than pregnancy. A version of this argument gets framed as 'clinical significance' vs. 'statistical significance,' which I think is useful differentiation. But as far as consumer products go, I left science half-a-decade ago and now build products for real people. I'm on the side of the layperson. You knew why I was taking this test, why I bought your product. I wanted to know if there was going to be a baby in ten months. Pick a better indicator of pregnancy. Design a better test. Take responsibility for your product. Stop blaming the user.

This definition is also contentious within the infectious disease community. For one, a person can be infected with a virus, or the virus can be present in their nose without being sick or contagious, particularly if they are partially immune. The ‘but technically’ crowd will say that it is not a false-positive— the virus was detected accurately. There might be a chance of spread, in the way there might be a chance the sun explodes tomorrow. But to real people living actual lives, if they are not contagious and not sick and would never become sick, they are healthy. I was taking this test to know if it was safe to go to my friend's birthday. It is as safe as life ever can be. That makes it a real false-positive. Stop lying to the user.

Debates about where to draw the line drawn between a clinical false-positive and a technical false positive usually become obnoxious, philosophical, or both. The best answer depends on the use case or inference being drawn from the data. No matter where one draws the line — whether it is cancer, COVID-19, or metal detectors — it does not matter to the broader overdiagnosis problem. There only needs errors of any kind for the problem to recur from screening or exhaustive search. There are infinite potential sources of noise given any standard or definition.