Home            Past Issues            About IP            About IAP           Author Information            Subscription            Advertisement              Search  

   
essentials of biostatistics

Indian Pediatr 2020;57: 43-48

Reporting of Basic Statistical Methods in Biomedical Journals: Improved SAMPL Guidelines

 

Abhaya Indrayan

From Department of Clinical Research, Max Healthcare, Saket, New Delhi, India.

Correspondence to: Dr Abhaya Indrayan, A-037 Telecom City, B-9/6 Sector 62, NOIDA 201 309, India.
Email: [email protected]

 

 

Statistical methods have become an essential component of all empirical biomedical research. Science requires that these methods are fully reported with complete accuracy so that the evidence base could be fully appraised for validity, reliability, and generalizability. To meet this objective, Statistical Analyses and Methods in Published Literature (SAMPL) guidelines have been prepared for statistical reporting in biomedical publications. This communication proposes substantial improvement of these guidelines to make them more comprehensive, organized, compact, and easier to adopt.

Keywords: Basic statistics, Guidelines, Statistical errors, Survival analysis.



Reporting of research is done to apprise others of the new development. This objective is more effectively achieved when the communi-cation contains enough details of the methodology and all other aspects so that the reader is convinced about the validity of the results, can assess their generalizability, and is able to replicate the results if needed.

Statistical methods have become an essential component of all empirical research publications, more so for biomedical research that confronts enormous uncertainties due to biological and environmental variability, sampling fluctuations, epistemic bottlenecks, and biases. Science requires that these methods are fully reported with complete accuracy so that the results could be fully appraised for validity, reliability and generalizability, and evidence-based medicine is strengthened. To meet this objective, Statistical Analyses and Methods in Published Literature (SAMPL) guidelines [1] have been prepared for statistical reporting in biomedical publications. However, these guidelines have some lacunae. For example, these guidelines mentioned about identifying the variables separately for Primary Analysis, for Reporting Hypothesis Tests, for Reporting Association Analysis, for Reporting Regression Analysis, and several others. Some of the essentials such as comparability of groups and robustness have been missed. These guidelines need to be reorganized on the lines of other reporting guidelines such as CONSORT. This communication proposes substantial improvement of these guidelines to make them more organized, compact, and easier to adopt.

Errors in Medical Research

Errors commonly creep into medical research endeavors, sometimes leading to false results [2-7]. Ioannidis [3] has expressed near inevitability of some false conclusions and has suggested designs to increase the chances of producing true results. PLoS Medicine editors [4] have opined that those involved in publication of research must make all efforts to reduce the chance of false conclusions. While some of this malaise can be attributed to the inappropriate methodology and questionable practices used in empirical research [5], some can be traced to poor reporting [7] that can happen even with otherwise good quality research. These deficiencies often render published results unusable [1,8]. Guidelines such as CONSORT, STROBE and STARD [9] have been developed for improved reporting of medical studies with different designs in the hope that adhering to these guidelines would reduce the chance of occurrence of these errors.

Statistical Errors

Many of the research errors are statistical in nature such as in design, elicitation of data, their processing and analysis, and the interpretation of the results [10-15]. Altman and Bland [16] in 1991 estimated that more than 50% papers at that time had some statistical errors and Wullschleger, et al. [17] found 64% (of a total 441) articles published in 2012 in three prime cardiovascular journals had inappropriate use of standard error of mean. Such errors often go unnoticed by the readers [18]. Sometimes, these errors can result in a statements that can jeopardize life and health of many people in course of time when inadequately substantiated result is used to treat millions of patients [10]. This is accentuated when the future research is built on the existing inadequately proven results. Techniques to avoid such statistical errors have been described earlier [19,20].

Performing the appropriate analysis is different from accurately describing it, and there is no way for a third person to assess what was actually adopted except by reporting in the publications. It is expected that much of these errors can be avoided by improved statistical reporting.

Statistical Reporting

Statistical reporting in biomedical publication is an important part of the Material and Methods section but it also affects the way the results are understood and interpreted. Several studies have observed that the statistical reporting in some biomedical publications is inadequate [11-14]. These studies suggest that this inadequacy generally occurs at three levels: (i) incomplete reporting leaving out room for readers to impute guess: (ii) willful or inadvertent erroneous reporting that has potential to arouse suspicion about the results; (iii) and inadequate interpretation of the statistical results. Much of this deficiency can be effectively addressed if the publications adhere to a standard guideline of items for reporting of the statistical methodology so that it is fully reported in a proper manner without missing any essential component. This may also encourage researchers to use the right statistical methods at various stages of their research.

Much of the clarity in reporting comes from clear statements about how the data were collected; what analysis was done how; why that particular analysis was appropriate for the problem in hand; and how the conclusion was drawn. Statistical methods in an empirical research can be intricate multivariate and multilevel analyses or can be specialized such as time series analysis whose description is admittedly challenging, but many errors have been observed in basic methods used in biomedical publications [21]. As these are basic methods, there is a tendency to use and describe them without sufficient care [22]. The proposed guidelines are focused on these basic methods only.

Guidelines for Statistical Reporting

In view of the common occurrence of statistical errors in biomedical publications, attempts have been made in the past to present guidelines for statistical reporting [17,23,24]. Subsequently, Lang and Altman [1] compiled a set of guidelines for basic statistical reporting for articles published in biomedical journals. They called it "Statistical Analyses and Methods in the Published Literature" or the SAMPL Guidelines, and these are now part of the EQUATOR network [9]. The authors acknowledge that these guidelines are limited to the basic methods but consider them sufficient to prevent most of the reporting deficiencies as the basic methods are also the most commonly used methods. The first guiding principle for these guidelines is that the statistical methods should be described with sufficient detail for a knowledgeable reader to verify the reported results if the data are provided to him, and the second principle is to report enough details of the descriptive statistics from which other indicators such as relative risk and odds ratio are derived.

Besides that the current SAMPL guidelines have not included some of the basic methods such as comparability of groups and robustness of results,these are also repititive. They also need to be reorganized in a compact form just as are other statements such as CONSORT, STROBE and STARD. These statements have been revised from time to time as and when new knowledge is acquired and it is time to revise the SAMPL guidelines as well to make them more organized, compact and easy to adopt. We have undertaken this exercise and the guidelines have been substantially revised in content and format (Table I). Most notable change is the complete reorganization of format to a numbered list for easy adoption. This also removes much of duplication. Other notable changes are inclusion of background information of the subjects, reporting of standardized rates (where needed) for comparability, robustness of results, not reporting mean and SD for extremely small sample size, and careful reporting of cause-effect inference. There are several other changes to make the guidelines more comprehensive and easy to understand. The reorganization is in terms of a list with 16 items, many with sub-items, which can also be used as a checklist. First 13 items will be required by almost any biomedical publication based on empirical data and the remaining 3 items are for specialized methods. To avoid duplication, there is no separate item on ANOVA and ANCOVA as reporting of these is included in other items. Bayesian analysis is also excluded as it is not a commonly used method in biomedical publications. Hazard ratio is excluded because of its specialized nature. Now, there is a clear demarcation of items to be reported for each analysis undertaken by the researcher although we continue to adhere to the principles enunciated earlier [1].

TABLE I Improved SAMPL Guidelines for Reporting Basic Statistical Methods in Biomedical Publications 
Topic No. Item
Subjects under study 1 Identify the target population, state the method of selection of the sample, total sample size, stratification if any, and the groups under study.
Sample size 2a State the sample size for each group and justify the size for the stated precision, alpha error, and/or power. For power, specify the smallest effect size considered medically important with reasons.
2b State the number of missing values, outliers and other exclusions with reasons, comment on the representativeness of the sample finally available for analysis, and describe possible biases with measures taken to control them.
Hypothesis 3a State all the hypotheses keeping the study objectives in mind.
3b State the minimum effect size to be considered as medically important, if applicable, with its rationale (see Item 1b). For equivalence and non-inferiority studies, give the largest medically unimportant margin with reasons.
Variables under study 4a State all the variables on which the data were collected and identify the ones on which the present analysis was done along with the rationale of the choice of variables. State the unit of measurement of each, and describe the validity of the methods of measurement for each variable.
4b Categorize continuous data for presentation of distribution if needed. If helpful, give histogram and comment on the distribution pattern, particularly of the outcome variables.
4c If dichotomous or polytomous categories have been used in analysis of continuous variables, explain the rationale of these categories in terms of clinical implication.
Antecedents and outcomes 5a In the case of analytical studies, identify the antecedent factors under study, the outcomes of interest, and the covariates included.
5b Define the effect of interest in terms of the variables included in the study (the effect size can be difference between means or between proportions, odds ratio, correlation coefficient, phi coefficient, or any other measure).
Descriptive summaries 6 Summarize the data –Provide mean (SD) (and not mean ± SD) or median (IQR) of each continuous variable depending upon the Gaussian or (highly) skewed distribution, respectively (do not use SE here). For IQR, give the values of the first and third quartile. Do not give such summaries for groups with n £4; give the original values instead. For categorical data, state actual frequency in different categories and the percentage if n ³20. All summaries should be with the appropriate degree of decimal accuracy as specified at the end of these guidelines*.
Modification of raw data 7 Describe transformation such as log and square-root, if any, with reasons and the method of calculation of scores, and rates and ratios, and fully specify the numerator, denominator and multiplier (per cent, per million, etc.) for each where applicable. For rates, specify the time period (per day, per year, etc.).
Baseline information 8 Summarize all important demographic and clinical features of the subjects in each group, particularly those that can affect the outcome (see Item 6).
Comparability of two or 9 Before comparing two or more groups with respect to outcomes in terms of summaries such as
more  groups means, proportions in different categories, and rates, confirm that the groups are comparable with regard to the baseline composition of the subjects for factors (such as the age distribution) that can affect the outcome. If not comparable, report the re-computed summaries after proper standardization. If standardization required but not done, state reasons and explain how the outcomes in various groups can still be compared.
Main method of analysis 10a Describe the method for each analysis, confirm the validity of the underlying assumptions, and justify the parametric and non-parametric methods used for different variables. Provide reference or explain the methods not in common use. State the software used for analysis with version.
10b Identify post-hoc analysis if any, including sub-groups analysis, and interpret this as exploratory and not confirmatory.
Estimation 11 For descriptive part of the study, provide estimate of the mean, proportion, difference, etc. with 95% confidence interval (CI). Justify the Gaussian approximation in case this is used for computing the CI. In case any other confidence level is used, provide the rationale.
Tests of statistical hypothesis 12a State the statistical hypothesis for each test. Give the name of each test and its exact P-value with df where relevant. For P<0.001, state with less than sign and for P>0.999 with more than sign. Indicate whether the test is one-tailed or two-tailed with the reasons thereof. Avoid the use of the term statistical significance and do not mention significance level (such as a = 0.05) for your results. Mention about any adjustment made for multiple comparisons and for using multiple tests for any conclusion. Distinguish between family-wise error rate and experiment-wise error rate. Also mention the CI for the effect size such as mean difference between the groups.
12b Report all the results and not just those that have low P-value. Interpret larger P-value as inconclusive and not as negative result unless the power is high to detect a specified medically important effect. Distinguish between results with low P-value (conventional statistical significance) and medical significance of the results.
Robustness of results 13 Comment about the statistical limitations of the study in addition to the other limita-tions. Statistical limitations could be due to imprecision of the measurements, restricted analysis because of the nature of the data or size of sample in different groups, not fulfilling the underlying assumptions, lack of representativeness of the sample, compromised design, lack of internal or external validations, and such other deficiencies.
The following are needed if these methods have been used in your paper
Correlation and cause-effect 14a Report the value of the relevant correlation coefficient. If described as low, moderate or high, give the categories with their biological implications. Interpret conventional Pearson correlation coefficient for assessing linear relationship and not for any general relationship between continuous variables. For association between categorical variables, include the full contingency table and explain if any categories were merged for analysis purpose.
14b Distinguish between association/correlation and cause-effect. If cause-effect is implied, rule out all possible alternative explanations such as the role of confounders and biases.
14c Distinguish correlation/association from agreement.
Regression analysis 15a Describe the purpose of the regression analysis (explanatory or predictive), identify the response (outcome) and regressor (antecedent) variables with the selection process if any, assess colinearity between independent variables, and provide medical and statistical rationale of the chosen model (linear/nonlinear, simple/multivariable). State the size of sample available for running each regression and comment on its adequacy. In case the model is being used for prediction of individual values, give prediction interval and not the CI for mean. Do not predict for values much beyond the values actually studied.
15b Report the regression equation with comments on its adequacy based on indicators such as coefficient of determination (h2, whose linear component is R2) for quantitative and generalized R2 for logistic regression, and report exact P-value for each regression coefficient with the associated CI. For quantitative dependent in simple linear or curvilinear regression, plot the regression line or curve with scatter where helpful and comment on the randomness of the residuals. For logistic regression, specify the reference category for categorical regressors, give odds ratio (OR) and the CI for each variable – adjusted as well as unadjusted. For cohort studies, state the number of subjects with positive and negative outcomes, and the relative risk with their CI – again adjusted as well as unadjusted. In the case of multivariable regression, interpret regression coefficient as adjusted only for the other variables in the model and give plausible biological explanation of the model obtained.
15c Specify whether and how the model was validated, or why it could not be validated.
Survival analysis 16a Describe the purpose of the survival analysis, identify the beginning- and the end-point for the duration under study, specify censoring, name the survival analysis method with the confirmation of the assumptions, plot the survival curve and report the median survival time with the CI, and discuss the points of inflexion in the survival curve, if relevant.
16b Where helpful, give the table with the estimated survival probability at each follow-up with the CI.
16c Specify the method used for comparing two or more survival curves if applicable and give exact P-value. Interpret it for overall survival pattern and not for specific time-points.

Decimal accuracy (rounded) as follows

Percentages - One decimal place if n <100 and two decimal places for n ³100;
Mean and SD (Median and IQR) - One decimal place more than the original values;
Correlation coefficient - Generally two decimal places;
Odds ratio, relative risk and hazard ratio - Generally two decimal places;
P-values - Exact P-values to three decimal places and not as P <0.05 or P
³0.05 (For extremely small values, write P <0.001, and for extremely high values, write P >0.999).

This revision is also restricted to the reporting of the basic methods. The advanced methods such as Cox regression, cluster analysis, and multivariate analysis of variance (MANOVA) are excluded in the hope that a qualified biostatistician will be involved when such advanced methods are used and the reporting will be adequate. The basic methods covered by these guidelines are generally used by those also who use advanced methods. To retain the focus, other methodological aspects such as design, allocation and randomization as well as issues relating to proper graphs, diagrams and tables are excluded. These suggested guidelines continue to be described in a manner that a statistically literate medical researcher can adopt without much help of a statistician. As in the case of original version [1], this suggested revision too is not prepared by a ‘formal consensus-building process’ but is prepared after consulting various other guidelines [24-27].

Conclusion

We hope that the editors of the biomedical journals will incorporate these guidelines in their instructions so that the reporting of basic statistical methods can improve and evidence-based results are reported. The real solution to poor statistical reporting will come when authors and statisticians learn more about research methodology and appropriate analysis, and also learn to communicate it properly [11]. Deficient statistical reporting underscores the need to expose the medical researchers to detailed texts [28,29] and structured biostatistics courses so that the methodology and reporting can improve.

Funding: None; Competing interest: None stated.

References

1. Lang T, Altman D. Basic Statistical Reporting for Articles Published in Clinical Medical Journals: The SAMPL Guidelines. In: Smart P, Maisonneuve H, Polderman A (eds). Science Editors’ Handbook. European Association of Science Editors, 2013.

2. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86-9.

3. Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2: e124.

4. The PLoS Medicine Editors. Minimizing mistakes and embracing uncertainty. PLoS Med. 2005;2:e272.

5. Golder WA. [Systematic errors in clinical studies: A comprehensive survey]. Z Rheumatol. 2017;76:71-82. (Article in German).

6. Baerlocher MO, O’Brien J, Newton M, Gautam T, Noble J. Data integrity, reliability and fraud in medical research. Eur J Intern Med. 2010;21:40-5.

7. Altman DG. Poor-quality medical research: What can journals do? JAMA. 2002;287:2765-7.

8. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383:267-76.

9. EQUATOR Network. Enhancing QUAlity and Transparancy of Health Research. Available from: https://www.equator-network.org/reporting-guidelines. Accessed January 9, 2019.

10. Indrayan A. Statistical fallacies and errors can also jeopardize life and health of many. Indian J Med Res. 2018;148;677-9.

11. Lang T. Twenty statistical errors even YOU can find in biomedical research articles. Croatian Med J. 2004; 45:361-70.

12. Strasak AM, Zaman Q, Pfeiffer KP, Göbel G, Ulmer H. Statistical errors in medical research–a review of common pitfalls. Swiss Med Wkly. 2007;137:44-9.

13. Shen J, Gou L, Tang J. Analysis of common on statistical errors in biomedical journals]. Zhongguo Xiu Fu Chong Jian Wai Ke Za Zhi. 2007;21:541-3.

14. Lee S. Avoiding negative reviewer comments: Common statistical errors in anesthesia journals. Korean J Anesthesiol. 2016;69:219-26.

15. Yan F, Robert M, Li Y. Statistical methods and common problems in medical or biomedical science research. Int J Physiol Pathophysiol Pharmacol. 2017;9:157-63.

16. Altman D, Bland JM. Improving doctors’ understanding of statistics. J Royal Stat Soc Series A (Statistics in Society). 1991;154:223-67.

17. Wullschleger M, Aghlmandi S, Egger M, Zwahlen M. High incorrect use of the standard error of the mean (SEM) in original articles in three cardiovascular journals evaluated for 2012. PLoS One. 2014;9:e110364.

18. Lang T, Secic M. How to Report Statistics in Medicine: Annotated Guidelines for Authors, Editors, and Reviewers, 2nd edition. Philadelphia: American College of Physicians, 2006.

19. Sullivan LM, Weinberg J, Keaney JF Jr. Common statistical pitfalls in basic science research. J Am Heart Assoc. 2016;5. pii: e004142.

20. Worthy G. Statistical analysis and reporting: Common errors found during peer review and how to avoid them. Swiss Med Wkly. 2015;145:w14076.

21. Dexter F. Checklist for statistical topics in Anesthesia and Analgesia reviews. Anesth Analg. 2011;113: 216-9.

22. Good PI, Hardin JW. Common Errors in Statistics (and How to Avoid Them), Fourth Edition. Wiley, 2012.

23. Altman DG, Gore SM, Gardner MJ, Pocock SJ. Statistical guidelines for contributors to medical journals. Br Med J (Clin Res Ed). 1983;286:1489-93.

24. Curran-Everett D, Benos DJ. Guidelines for reporting statistics in journals published by the American Physio-logical Society: The sequel. Adv Physiol Educ. 2007;31: 295-8.

25. Lindsey ML, Gray GA, Wood SK, Curran-Everett D. Statistical considerations in reporting cardiovascular research. Am J Physiol Heart Circ Physiol. 2018;315: H303-H313.

26. Hickey GL, Dunning J, Seifert B, Sodeck G, Carr MJ, Burger HU, et al.; EJCTS and ICVTS Editorial Committees. Statistical and Data Reporting Guidelines for the European Journal of Cardio-Thoracic Surgery and the Interactive CardioVascular and Thoracic Surgery. Eur J Cardiothorac Surg. 2015;48:180-93.

27. Hutton B, Wolfe D, Moher D, Shamseer L. Reporting guidance considerations from a statistical perspective: Overview of tools to enhance the rigour of reporting of randomised trials and systematic reviews. Evid Based Ment Health. 2017;20:46-52.

28. Indrayan A, Malhotra RK. Medical Biostatistics, 4th Edition. Chapman and Hall/ CRC Press, 2018.

29. Dawson B, Trapp RG. Basic and Clinical Biostatistics, Fourth Edition. McGraw Hill, 2004.

 

Copyright © 1999-2020  Indian Pediatrics