Is it time to stop using statistical significance?

An over-reliance on statistical significance and p values may lead to incorrect conclusions when evaluating clinical evidence. In the February edition of Australian Prescriber, Senior Research Fellow Dr Oliver Frank from the University of Adelaide Medical School and co-authors discuss why statistical significance should not be used.

“An effect being statistically significant does not imply that it is clinically significant,” says Dr Frank.

“For example, a study could find that one drug is better than another with a high level of statistical significance, but the effect is so small that it is below the threshold of what is agreed to be clinically relevant,” says Dr Frank.

Factors that are more important than statistical significance include appropriate trial design and research methods.

The Australian Prescriber article also considers confidence intervals to be important when evaluating clinical evidence.

“If the clinical interpretation is the same for all values of a parameter inside its confidence interval, a conclusion can be drawn with confidence. Interpretations using parameters with wide confidence intervals should be taken with a grain of salt,” says Dr Frank.

Read the article in Australian Prescriber.