Jonas Ranstam's website
Jonas Ranstam, BSc PhD, is a former professor of medical statistics at Lund University, Sweden, and an internationally active medical statistical consultant with extensive experience from collaboration in randomised trials and epidemiological studies. He has long had a particular interest in statistical reviewing, clinical trials, and chronic disease epidemiology and was in 2016 the overall winner of Publons' Sentinels of Science Awards. Of the more than 3 million researchers currently registered at Publons, Jonas Ranstam is still the most experienced reviewer.
Statistical inference and significance
During the last 100 years, the ambition behind medical research publications has grown from describing subjective opinions to presenting objective evidence. Statistical measures such as p-values and statistical significance now play leading roles in most research reports, but the overall scientific reasoning is often surprisingly poor. Methodological misconceptions are ubiquitous, which makes statistical reviewing an essential part of medical research.
The fundamental problem to be solved using statistical inference is sampling variation, i.e. multiple samples drawn from the same population have varying characteristics, which causes uncertainty when studying just one sample (e.g. the patients in a randomised trial or an observational cohort). The uncertainty needs to be quantified to draw rational conclusions from the observations. It is usually presented in terms of p-values and confidence intervals.
The role of the p-value is to indicate the compatibility between observed data and a specific statistical model. P-values are thus relevant for a medical investigation only if the data are accurate and the models adequate, which implies that sound scientific argumentation requires more statistical reasoning than just presenting p-values. For example, the investigator should explain why the studied data are accurate and how various sources of bias have been accounted for in the study design and the statistical analysis. The statistical models underlying referenced p-values need to be presented and motivated. Clinically relevant differences need to be defined, and the estimation uncertainty (indicated by confidence intervals) evaluated when interpreting the effect size of a finding.
In practice, confidence intervals are often misinterpreted as dispersion measures, and complex research problems are 'solved' by simplistically calculating p-values and dichotomising them into significant (p<0.05) and nonsignificant (p>0.05). The former is incorrectly presented as evidence of a practically important finding and the latter as evidence of equivalence or 'no difference'. Both interpretations are flawed. Data collection and study design are often considered unimportant. Data collection and study design are often considered irrelevant. The p-value has become a substitute for scientific reasoning.
Banning p-values, as done by some scientific journals, may be one way to get rid of misused p-values but the problem of sampling variation remains to be solved. A good investigator uses statistical reasoning to evaluate how the limitations of sampling, data collection, study design, and statistical analysis affect the outcomes of an investigation. The evaluation requires statistical inference, and it can be performed with and without p-values.
Jonas Ranstam is a member of ISCB 🔗 and ISI 🔗.