By Grant Currin
In 2016 Nicole Lazar, then the editor of The American Statistician, received a strange email from the executive director of the American Statistical Association. It read, “I need to talk to you about something but I don’t want to put it in email. Can I call you?”
“I thought ‘uh oh, what have I done?’” Lazar told an audience at a ScienceWriters2019 session titled "The end of statistical significance".
Earlier in the year, the ASA had convened a committee to discuss the abuse and misuse of the p-value, which is widely used to indicate the likelihood that the result of hypothesis testing is statistically significant.
The threshold p < .05 is accepted in many areas of science as a bright line that clearly indicates whether a null hypothesis should be rejected, but the arbitrary threshold is falling out of favor as a means of determining truth in science.
Statisticians and scientists have been questioning and criticizing the p-value for decades, and recent problems in replicating and reproducing headline-grabbing studies in neuroscience, social psychology, cancer research, and others have drawn more attention to the problem.
After consultation with Lazar, the ASA published “The ASA Statement on p-Values: Context, Process, and Purpose,” in The American Statistician in March, 2016. In addition to the statement itself, Lazar solicited commentary from individual members of the committees. It was the rare blockbuster statistics paper: the statement octupled The American Statistician’s impact factor and as of November 2019, the paper has been viewed more than 350,000 times.
“For the most part, people are not saying do away with p-values,” she said. Most statisticians just want scientists to use them differently.
Here are some key points for science writers to consider going forward:
There’s no one-size-fits-all solution
Statisticians think that different disciplines are going to come to different solutions. “And that’s okay,” Lazar said.
A move away from the standardized statistics that have dominated the sciences for decades will force science writers to understand and interpret more types of statistical models and conclusions, but such a move is necessary for the advancement of science.
“The sciences are different, and they have different meanings,” she said. What makes sense in one field might not be the best fit in another.
Accept uncertainty
One big problem with p-value thresholds is that they can be used to “impose an artificial certainty,” Lazar said. Through manipulation, misinterpretation, or just random chance, p-value thresholds can show clarity or simplicity where it doesn’t exist. Instead, Lazar advises embracing uncertainty and looking beyond the “significant” or “highly significant” labels.
“You have to switch your way of thinking, talking about data, and reporting statistical results,’ she said. “It’s hard.”
Be thoughtful, open, and modest
For scientists, using better statistics for better science means thoroughly reporting what they’ve done and including more information, not less. Lazar advises journalists to remember that one study does not tell the whole story.
“It’s converging evidence from multiple studies that’s going to lead us to true scientific insight,” she said. Stories or press releases that exaggerate the size of an effect or the importance of a claim can ultimately be harmful to researchers, institutions, and the areas of research where they’re made. Lazar recommends focusing instead on the substantive implications of research, whether it’s to an esoteric corner of basic science or an applied study that will quickly affect people’s lives.
Grant Currin is a Nashville-based freelance science writer and a recipient of a 2019 NASW Travel Fellowship. Follow him on Twitter @GrantCurrin.
