CTE, the NFL & Selection Bias

It’s not often a scientific article gets a spread like this in the New York Times, but it just goes to show - concussions get attention.

 

On July 25, 2017, an article was published in the Journal of the American Medical Association (JAMA) which has made its way through the popular media and sports media. If you haven’t been asked about this article yet, you probably will be. Here’s a link to the abstract so you can take a look for yourself. (I’d prefer to provide the full text, but unfortunately it’s not an open-access article.)

 

It comes down to the question being asked

The research paper is asking "What are the neuropathological and clinical features of a case series of deceased players of American football neuropathologically diagnosed as having chronic traumatic encephalopathy (CTE)?” In other words, they wanted to know what they could learn about CTE. 

On the other hand, the question being asked or implied by some (not all) of the popular media coverage of this research seems to be “Does football cause CTE?” 

The study authors want to know more about the brains and the people who have CTE. That’s extremely valuable, especially when we know so little about the disease. We don’t know what causes it, or who gets it, or why. We can diagnose it, but only on autopsy, so we can’t even tell if someone has it before they die. We can only suspect CTE if they have certain symptoms - but we can’t definitively differentiate it from mental illness like depression or other neurodegenerative diseases like dementia until after death. 

But the study question is a bit nuanced, and “Does football cause CTE?" is a better headline. When the data is framed as "110 of 111 football players have CTE,” people take notice.

Now, both questions are important - we need to know more about CTE, including what causes it. We just need to make sure that we understand the question the researchers were trying to answer. And the reporting of this story leaves out one big factor:

 

Selection bias

Selection bias refers to non-random selection of data or individuals in a study. Ideally, we want a sample of people randomly chosen from the population at large, so that we can say the results of the study represent that population. When the selection of participants is biased, it doesn’t represent the population at large, and it limits the conclusions we can draw from the data. 

The authors of this CTE know their sample is biased - they refer to their group of players as a “convenience sample.” This basically means they took what they could get, and when a study requires donated brains, this is pretty much the only way to go. The trouble is, the whole study population is people who were worried about CTE and chose to donate their brains to a CTE study. 

From this data, we can say that of 111 NFL players who chose to donate their brains, 110 had CTE. We can’t say that 99.09% of NFL players will get CTE - the study wasn’t set up that way. 

 

Why football?

The authors are, however, able to conclude that “CTE may be related to prior participation in football.” We don’t know why football might be a problem, because we don’t really understand what leads to CTE. Does CTE happen to people who have had lots of concussions? (And how many is “lots,” anyways?) Or does CTE happen to people who get lots of little hits, but never have symptoms of a concussion (known as sub-concussive impacts)? If we can answer the question about what leads to CTE, we can probably identify other groups at risk of the disease. 

 

The take-home message

This study presents a lot of really great demographic, health history, and neuropathological data about brains that are shown to have CTE. An awful lot of those brains came from former football players, and those who played at higher levels had more severe pathology. It doesn’t mean that every player you watch on Sunday Night Football will have CTE in ten years, but it’s pretty huge, and it should make people in football organizations sit up and take notice. As for drawing other conclusions, as with any good study, this one has pointed out that more research is needed.