essentials of biostatistics |
|
Indian Pediatr 2020;57: 43-48 |
|
Reporting of Basic Statistical Methods in
Biomedical Journals: Improved SAMPL Guidelines
|
Abhaya Indrayan
From Department of Clinical Research, Max Healthcare,
Saket, New Delhi, India.
Correspondence to: Dr Abhaya Indrayan, A-037 Telecom
City, B-9/6 Sector 62, NOIDA 201 309, India.
Email: [email protected]
|
Statistical methods have become an
essential component of all empirical biomedical research. Science
requires that these methods are fully reported with complete accuracy so
that the evidence base could be fully appraised for validity,
reliability, and generalizability. To meet this objective, Statistical
Analyses and Methods in Published Literature (SAMPL) guidelines have
been prepared for statistical reporting in biomedical publications. This
communication proposes substantial improvement of these guidelines to
make them more comprehensive, organized, compact, and easier to adopt.
Keywords: Basic statistics, Guidelines,
Statistical errors, Survival analysis.
|
Reporting of research is done to apprise others of the new development.
This objective is more effectively achieved when the communi-cation
contains enough details of the methodology and all other aspects so that
the reader is convinced about the validity of the results, can assess
their generalizability, and is able to replicate the results if needed.
Statistical methods have become an essential
component of all empirical research publications, more so for biomedical
research that confronts enormous uncertainties due to biological and
environmental variability, sampling fluctuations, epistemic bottlenecks,
and biases. Science requires that these methods are fully reported with
complete accuracy so that the results could be fully appraised for
validity, reliability and generalizability, and evidence-based medicine
is strengthened. To meet this objective, Statistical Analyses and
Methods in Published Literature (SAMPL) guidelines [1] have been
prepared for statistical reporting in biomedical publications. However,
these guidelines have some lacunae. For example, these guidelines
mentioned about identifying the variables separately for Primary
Analysis, for Reporting Hypothesis Tests, for Reporting Association
Analysis, for Reporting Regression Analysis, and several others. Some of
the essentials such as comparability of groups and robustness have been
missed. These guidelines need to be reorganized on the lines of other
reporting guidelines such as CONSORT. This communication proposes
substantial improvement of these guidelines to make them more organized,
compact, and easier to adopt.
Errors in Medical Research
Errors commonly creep into medical research
endeavors, sometimes leading to false results [2-7]. Ioannidis [3] has
expressed near inevitability of some false conclusions and has suggested
designs to increase the chances of producing true results. PLoS
Medicine editors [4] have opined that those involved in publication
of research must make all efforts to reduce the chance of false
conclusions. While some of this malaise can be attributed to the
inappropriate methodology and questionable practices used in empirical
research [5], some can be traced to poor reporting [7]
that can happen even with otherwise good quality
research. These deficiencies often render published results unusable
[1,8]. Guidelines such as CONSORT, STROBE and STARD [9] have been
developed for improved reporting of medical studies with different
designs in the hope that adhering to these guidelines would reduce the
chance of occurrence of these errors.
Statistical Errors
Many of the research errors are statistical in nature
such as in design, elicitation of data, their processing and analysis,
and the interpretation of the results [10-15]. Altman and Bland [16] in
1991 estimated that more than 50% papers at that time had some
statistical errors and Wullschleger, et al. [17]
found 64% (of a total 441) articles published in
2012 in three prime cardiovascular journals had inappropriate use of
standard error of mean. Such errors often go unnoticed by the readers
[18]. Sometimes, these errors can result in a statements that can
jeopardize life and health of many people in course of time when
inadequately substantiated result is used to treat millions of patients
[10]. This is accentuated when the future research is built on the
existing inadequately proven results. Techniques to avoid such
statistical errors have been described earlier [19,20].
Performing the appropriate analysis is different from
accurately describing it, and there is no way for a third person to
assess what was actually adopted except by reporting in the
publications. It is expected that much of these errors can be avoided by
improved statistical reporting.
Statistical Reporting
Statistical reporting in biomedical publication is an
important part of the Material and Methods section but it also affects
the way the results are understood and interpreted. Several studies have
observed that the statistical reporting in some biomedical publications
is inadequate [11-14]. These studies suggest that this inadequacy
generally occurs at three levels: (i) incomplete reporting
leaving out room for readers to impute guess: (ii) willful or
inadvertent erroneous reporting that has potential to arouse suspicion
about the results; (iii) and inadequate interpretation of the
statistical results. Much of this deficiency can be effectively
addressed if the publications adhere to a standard guideline of items
for reporting of the statistical methodology so that it is fully
reported in a proper manner without missing any essential component.
This may also encourage researchers to use the right statistical methods
at various stages of their research.
Much of the clarity in reporting comes from clear
statements about how the data were collected; what analysis was done
how; why that particular analysis was appropriate for the problem in
hand; and how the conclusion was drawn. Statistical methods in an
empirical research can be intricate multivariate and multilevel analyses
or can be specialized such as time series analysis whose description is
admittedly challenging, but many errors have been observed in basic
methods used in biomedical publications [21]. As these are basic
methods, there is a tendency to use and describe them without sufficient
care [22]. The proposed guidelines are focused on these basic methods
only.
Guidelines for Statistical Reporting
In view of the common occurrence of statistical
errors in biomedical publications, attempts have been made in the past
to present guidelines for statistical reporting [17,23,24].
Subsequently, Lang and Altman [1] compiled a set of guidelines for basic
statistical reporting for articles published in biomedical journals.
They called it "Statistical Analyses and Methods in the Published
Literature" or the SAMPL Guidelines, and these are now part of the
EQUATOR network [9]. The authors acknowledge that these guidelines are
limited to the basic methods but consider them sufficient to prevent
most of the reporting deficiencies as the basic methods are also the
most commonly used methods. The first guiding principle for these
guidelines is that the statistical methods should be described with
sufficient detail for a knowledgeable reader to verify the reported
results if the data are provided to him, and the second principle is to
report enough details of the descriptive statistics from which other
indicators such as relative risk and odds ratio are derived.
Besides that the current SAMPL guidelines have not
included some of the basic methods such as comparability of groups and
robustness of results,these are also repititive. They also need to be
reorganized in a compact form just as are other statements such as
CONSORT, STROBE and STARD. These statements have been revised from time
to time as and when new knowledge is acquired and it is time to revise
the SAMPL guidelines as well to make them more organized, compact and
easy to adopt. We have undertaken this exercise and the guidelines have
been substantially revised in content and format (Table I).
Most notable change is the complete reorganization of format to a
numbered list for easy adoption. This also removes much of duplication.
Other notable changes are inclusion of background information of the
subjects, reporting of standardized rates (where needed) for
comparability, robustness of results, not reporting mean and SD for
extremely small sample size, and careful reporting of cause-effect
inference. There are several other changes to make the guidelines more
comprehensive and easy to understand. The reorganization is in terms of
a list with 16 items, many with sub-items, which can also be used as a
checklist. First 13 items will be required by almost any biomedical
publication based on empirical data and the remaining 3 items are for
specialized methods. To avoid duplication, there is no separate item on
ANOVA and ANCOVA as reporting of these is included in other items.
Bayesian analysis is also excluded as it is not a commonly used method
in biomedical publications. Hazard ratio is excluded because of its
specialized nature. Now, there is a clear demarcation of items to be
reported for each analysis undertaken by the researcher although we
continue to adhere to the principles enunciated earlier [1].
TABLE I Improved SAMPL Guidelines for Reporting Basic Statistical Methods in Biomedical Publications
Topic |
No. |
Item |
Subjects under study |
1 |
Identify the target population, state the method of selection of
the sample, total sample size, stratification if any, and the
groups under study. |
Sample size |
2a |
State the sample size for each group and justify the size for
the stated precision, alpha error, and/or power. For power,
specify the smallest effect size considered medically important
with reasons. |
|
2b |
State the number of missing values, outliers and other
exclusions with reasons, comment on the representativeness of
the sample finally available for analysis, and describe possible
biases with measures taken to control them. |
Hypothesis |
3a |
State all the hypotheses keeping the study objectives in mind. |
|
3b |
State the minimum effect size to be considered as medically
important, if applicable, with its rationale (see Item 1b). For
equivalence and non-inferiority studies, give the largest
medically unimportant margin with reasons. |
Variables under study |
4a |
State all the variables on which the data were collected and
identify the ones on which the present analysis was done along
with the rationale of the choice of variables. State the unit of
measurement of each, and describe the validity of the methods of
measurement for each variable. |
|
4b |
Categorize continuous data for presentation of distribution if
needed. If helpful, give histogram and comment on the
distribution pattern, particularly of the outcome variables. |
|
4c |
If dichotomous or polytomous categories have been used in
analysis of continuous variables, explain the rationale of these
categories in terms of clinical implication. |
Antecedents and outcomes |
5a |
In the case of analytical studies, identify the antecedent
factors under study, the outcomes of interest, and the
covariates included.
|
|
5b |
Define the effect of interest in terms of the variables included
in the study (the effect size can be difference between means or
between proportions, odds ratio, correlation coefficient, phi
coefficient, or any other measure). |
Descriptive summaries |
6 |
Summarize the data –Provide mean (SD) (and not mean ± SD) or
median (IQR) of each continuous variable depending upon the
Gaussian or (highly) skewed distribution, respectively (do not
use SE here). For IQR, give the values of the first and third
quartile. Do not give such summaries for groups with n £4; give
the original values instead. For categorical data, state actual
frequency in different categories and the percentage if n ³20.
All summaries should be with the appropriate degree of decimal
accuracy as specified at the end of these guidelines*. |
Modification of raw data |
7 |
Describe transformation such as log and square-root, if any,
with reasons and the method of calculation of scores, and rates
and ratios, and fully specify the numerator, denominator and
multiplier (per cent, per million, etc.) for each where
applicable. For rates, specify the time period (per day, per
year, etc.). |
Baseline information |
8 |
Summarize all important demographic and clinical features of the
subjects in each group, particularly those that can affect the
outcome (see Item 6). |
Comparability of two or
|
9 |
Before comparing two or more groups with respect to outcomes in
terms of summaries such as
|
more groups
|
|
means, proportions in different categories, and rates, confirm
that the groups are comparable with regard to the baseline
composition of the subjects for factors (such as the age
distribution) that can affect the outcome. If not comparable,
report the re-computed summaries after proper standardization.
If standardization required but not done, state reasons and
explain how the outcomes in various groups can still be
compared. |
Main method of analysis |
10a |
Describe the method for each analysis, confirm the validity of
the underlying assumptions, and justify the parametric and
non-parametric methods used for different variables. Provide
reference or explain the methods not in common use. State the
software used for analysis with version. |
|
10b |
Identify post-hoc analysis if any, including sub-groups
analysis, and interpret this as exploratory and not
confirmatory. |
Estimation
|
11 |
For descriptive part of the study, provide estimate of the mean,
proportion, difference, etc. with 95% confidence interval (CI).
Justify the Gaussian approximation in case this is used for
computing the CI. In case any other confidence level is used,
provide the rationale. |
Tests of statistical hypothesis |
12a |
State the statistical hypothesis for each test. Give the name of
each test and its exact P-value with df where relevant. For
P<0.001, state with less than sign and for P>0.999 with more
than sign. Indicate whether the test is one-tailed or two-tailed
with the reasons thereof. Avoid the use of the term statistical
significance and do not mention significance level (such as a =
0.05) for your results. Mention about any adjustment made for
multiple comparisons and for using multiple tests for any
conclusion. Distinguish between family-wise error rate and
experiment-wise error rate. Also mention the CI for the effect
size such as mean difference between the groups. |
|
12b |
Report all the results and not just those that have low P-value.
Interpret larger P-value as inconclusive and not as negative
result unless the power is high to detect a specified medically
important effect. Distinguish between results with low P-value
(conventional statistical significance) and medical significance
of the results. |
Robustness of results |
13 |
Comment about the statistical limitations of the study in
addition to the other limita-tions. Statistical limitations
could be due to imprecision of the measurements, restricted
analysis because of the nature of the data or size of sample in
different groups, not fulfilling the underlying assumptions,
lack of representativeness of the sample, compromised design,
lack of internal or external validations, and such other
deficiencies. |
The following are needed if these methods have been used in
your paper |
Correlation and cause-effect |
14a |
Report the value of the relevant correlation coefficient. If
described as low, moderate or high, give the categories with
their biological implications. Interpret conventional Pearson
correlation coefficient for assessing linear relationship and
not for any general relationship between continuous variables.
For association between categorical variables, include the full
contingency table and explain if any categories were merged for
analysis purpose. |
|
14b |
Distinguish between association/correlation and cause-effect. If
cause-effect is implied, rule out all possible alternative
explanations such as the role of confounders and biases. |
|
14c |
Distinguish correlation/association from agreement. |
Regression analysis |
15a |
Describe the purpose of the regression analysis (explanatory or
predictive), identify the response (outcome) and regressor
(antecedent) variables with the selection process if any, assess
colinearity between independent variables, and provide medical
and statistical rationale of the chosen model (linear/nonlinear,
simple/multivariable). State the size of sample available for
running each regression and comment on its adequacy. In case the
model is being used for prediction of individual values, give
prediction interval and not the CI for mean. Do not predict for
values much beyond the values actually studied. |
|
15b |
Report the regression equation with comments on its adequacy
based on indicators such as coefficient of determination (h2,
whose linear component is R2) for quantitative and generalized
R2 for logistic regression, and report exact P-value for each
regression coefficient with the associated CI. For quantitative
dependent in simple linear or curvilinear regression, plot the
regression line or curve with scatter where helpful and comment
on the randomness of the residuals. For logistic regression,
specify the reference category for categorical regressors, give
odds ratio (OR) and the CI for each variable – adjusted as well
as unadjusted. For cohort studies, state the number of subjects
with positive and negative outcomes, and the relative risk with
their CI – again adjusted as well as unadjusted. In the case of
multivariable regression, interpret regression coefficient as
adjusted only for the other variables in the model and give
plausible biological explanation of the model obtained. |
|
15c |
Specify whether and how the model was validated, or why it could
not be validated. |
Survival analysis |
16a |
Describe the purpose of the survival analysis, identify the
beginning- and the end-point for the duration under study,
specify censoring, name the survival analysis method with the
confirmation of the assumptions, plot the survival curve and
report the median survival time with the CI, and discuss the
points of inflexion in the survival curve, if relevant.
|
|
16b |
Where helpful, give the table with the estimated survival
probability at each follow-up with the CI. |
|
16c |
Specify the method used for comparing two or more survival
curves if applicable and give exact P-value. Interpret it for
overall survival pattern and not for specific time-points. |
Decimal accuracy (rounded) as follows
Percentages - One decimal place if n <100 and
two decimal places for n ³100;
Mean and SD (Median and IQR) - One decimal place more than the
original values;
Correlation coefficient - Generally two decimal places;
Odds ratio, relative risk and hazard ratio - Generally two
decimal places;
P-values - Exact P-values to three decimal places and not as P
<0.05 or P ³0.05
(For extremely small values, write P <0.001, and for extremely
high values, write P >0.999).
|
This revision is also restricted to the reporting of
the basic methods. The advanced methods such as Cox regression, cluster
analysis, and multivariate analysis of variance (MANOVA) are excluded in
the hope that a qualified biostatistician will be involved when such
advanced methods are used and the reporting will be adequate. The basic
methods covered by these guidelines are generally used by those also who
use advanced methods. To retain the focus, other methodological aspects
such as design, allocation and randomization as well as issues relating
to proper graphs, diagrams and tables are excluded. These suggested
guidelines continue to be described in a manner that a statistically
literate medical researcher can adopt without much help of a
statistician. As in the case of original version [1], this suggested
revision too is not prepared by a ‘formal consensus-building process’
but is prepared after consulting various other guidelines [24-27].
Conclusion
We hope that the editors of the biomedical journals
will incorporate these guidelines in their instructions so that the
reporting of basic statistical methods can improve and evidence-based
results are reported. The real solution to poor statistical reporting
will come when authors and statisticians learn more about research
methodology and appropriate analysis, and also learn to communicate it
properly [11]. Deficient statistical reporting underscores the need to
expose the medical researchers to detailed texts [28,29] and structured
biostatistics courses so that the methodology and reporting can improve.
Funding: None; Competing interest: None
stated.
References
1. Lang T, Altman D. Basic Statistical Reporting for
Articles Published in Clinical Medical Journals: The SAMPL Guidelines.
In: Smart P, Maisonneuve H, Polderman A (eds). Science
Editors’ Handbook. European Association of Science Editors, 2013.
2. Chalmers I, Glasziou P. Avoidable waste in the
production and reporting of research evidence. Lancet. 2009;374:86-9.
3. Ioannidis JPA. Why most published research
findings are false. PLoS Med. 2005;2: e124.
4. The PLoS Medicine Editors. Minimizing mistakes and
embracing uncertainty. PLoS Med. 2005;2:e272.
5. Golder WA. [Systematic errors in clinical studies:
A comprehensive survey]. Z Rheumatol. 2017;76:71-82. (Article in
German).
6. Baerlocher MO, O’Brien J, Newton M, Gautam T,
Noble J. Data integrity, reliability and fraud in medical research. Eur
J Intern Med. 2010;21:40-5.
7. Altman DG. Poor-quality medical research: What can
journals do? JAMA. 2002;287:2765-7.
8. Glasziou P, Altman DG, Bossuyt P, Boutron I,
Clarke M, Julious S, et al. Reducing waste from incomplete or
unusable reports of biomedical research. Lancet. 2014;383:267-76.
9. EQUATOR Network. Enhancing QUAlity and
Transparancy of Health Research. Available from:
https://www.equator-network.org/reporting-guidelines. Accessed
January 9, 2019.
10. Indrayan A. Statistical fallacies and errors can
also jeopardize life and health of many. Indian J Med Res.
2018;148;677-9.
11. Lang T. Twenty statistical errors even YOU can
find in biomedical research articles. Croatian Med J. 2004; 45:361-70.
12. Strasak AM, Zaman Q, Pfeiffer KP, Göbel G, Ulmer
H. Statistical errors in medical research–a review of common pitfalls.
Swiss Med Wkly. 2007;137:44-9.
13. Shen J, Gou L, Tang J. Analysis of common on
statistical errors in biomedical journals]. Zhongguo Xiu Fu Chong Jian
Wai Ke Za Zhi. 2007;21:541-3.
14. Lee S. Avoiding negative reviewer comments:
Common statistical errors in anesthesia journals. Korean J Anesthesiol.
2016;69:219-26.
15. Yan F, Robert M, Li Y. Statistical methods and
common problems in medical or biomedical science research. Int J Physiol
Pathophysiol Pharmacol. 2017;9:157-63.
16. Altman D, Bland JM. Improving doctors’
understanding of statistics. J Royal Stat Soc Series A (Statistics in
Society). 1991;154:223-67.
17. Wullschleger M, Aghlmandi S, Egger M, Zwahlen M.
High incorrect use of the standard error of the mean (SEM) in original
articles in three cardiovascular journals evaluated for 2012. PLoS One.
2014;9:e110364.
18. Lang T, Secic M. How to Report Statistics in
Medicine: Annotated Guidelines for Authors, Editors, and Reviewers, 2nd
edition. Philadelphia: American College of Physicians, 2006.
19. Sullivan LM, Weinberg J, Keaney JF Jr. Common
statistical pitfalls in basic science research. J Am Heart Assoc.
2016;5. pii: e004142.
20. Worthy G. Statistical analysis and reporting:
Common errors found during peer review and how to avoid them. Swiss Med
Wkly. 2015;145:w14076.
21. Dexter F. Checklist for statistical topics
in Anesthesia and Analgesia reviews. Anesth Analg. 2011;113: 216-9.
22. Good PI, Hardin JW. Common Errors in Statistics
(and How to Avoid Them), Fourth Edition. Wiley, 2012.
23. Altman DG, Gore SM, Gardner MJ, Pocock SJ.
Statistical guidelines for contributors to medical journals. Br Med J (Clin
Res Ed). 1983;286:1489-93.
24. Curran-Everett D, Benos DJ. Guidelines for
reporting statistics in journals published by the American Physio-logical
Society: The sequel. Adv Physiol Educ. 2007;31: 295-8.
25. Lindsey ML, Gray GA, Wood SK, Curran-Everett D.
Statistical considerations in reporting cardiovascular research. Am J
Physiol Heart Circ Physiol. 2018;315: H303-H313.
26. Hickey GL, Dunning J, Seifert B, Sodeck G, Carr
MJ, Burger HU, et al.; EJCTS and ICVTS Editorial Committees.
Statistical and Data Reporting Guidelines for the European Journal of
Cardio-Thoracic Surgery and the Interactive CardioVascular and Thoracic
Surgery. Eur J Cardiothorac Surg. 2015;48:180-93.
27. Hutton B, Wolfe D, Moher D, Shamseer L. Reporting
guidance considerations from a statistical perspective: Overview of
tools to enhance the rigour of reporting of randomised trials and
systematic reviews. Evid Based Ment Health. 2017;20:46-52.
28. Indrayan A, Malhotra RK. Medical Biostatistics,
4th Edition. Chapman and Hall/ CRC Press, 2018.
29. Dawson B, Trapp RG. Basic and Clinical Biostatistics, Fourth
Edition. McGraw Hill, 2004.
|
|
|
|