|
Indian Pediatr 2011;48: 4 90-491 |
|
Negative Studies Published in Indian Medical
Journals are Underpowered |
Jaykaran, Deepak Saxena*, Preeti Yadav and NK Kantharia,
Department of Pharmacology and Community Medicine,*
Government Medical College, Surat, Gujarat, India.
Email:
[email protected]
|
51 inconclusive studies published in 14 Indian medical journals were
analyzed for adequacy of power by post hoc power calculation. No study
was found to be adequately powered (>80%) at small effect size, only 8
studies were adequately powered for medium effect size and 30 studies
were adequately powered at large effect size.
|
Inconclusive studies published in medical journals should have proper
reporting of power and sample size so that their validity can be assessed
by readers. A study may become inconclusive because of two main reasons
one there may be actually no difference between the groups for the
outcomes or sample size/power of study is not adequate to measure that
difference. Results of a study can only be considered valid when the study
is adequately powered to see the difference between the groups [1]. It has
been observed in the studies done for articles published in westerns
medical journals that many of the studies including inconclusive and
negative studies are not adequately powered to detect difference between
the groups [2,3]. Publications of these underpowered studies are
considered as unethical because they exposes the patients to adverse
effects of interventions but unable to detect which intervention is
superior [4,5]. Though many studies are published for assessment of
articles published in western medical journals for adequacy of power,
similar data are lacking for the Indian Medical Journals. So this study
was designed with the aim of critical evaluation of inconclusive studies
published in 14 Indian medical journals for adequacy of power by post hoc
power calculation.
All the Pubmed indexed journals subscribed by central
library of our institute were taken into consideration. These journals
were Indian Pediatrics (IP)(inconclusive studies 6/13), Annals of Indian
Academy of Neurology (AIAN)(5/13), Indian Journal of Orthopedics (IJ
Ortho) (inconclusive studies 7/63), Indian Journal of Critical Care
Medicine (IJCCM) (inconclusive studies 2/15), Indian Journal of
Dermatology, Vanereology and Leprology (IJDVL) (inconclusive studies
4/16), Indian Journal of Nephrology (IJN) (inconclusive studies 1/14),
Indian Journal of Dermatology (IJD) (inconclusive studies 3/21), Indian
Journal of Ophthalmology (IJO) (inconclusive studies 1/15), Indian Journal
of Urology (IJU) (inconclusive studies 8/24), Indian Journal of Anesthesia
(IJA) (inconclusive studies 13/46), Indian Journal of Psychiatry (IJ Psy)
(inconclusive studies 1/8), Indian Journal of Medical Research (IJMR)
(inconclusive studies 0/12), Indian Journal of Medical Science (IJMS)
(inconclusive studies 0/6) and Indian Journal of Community Medicine (IJCM)
(inconclusive studies 0/10). All the original articles published in 2009
from these journals were evaluated to identify inconclusive studies.
Studies were considered inconclusive studies on the basis of these
criteria-one: primary outcome is not statistically significant, second:
most important outcome is not statistically significant (if primary out
come is not reported). Post hoc power of all these inconclusive studies
were calculated with the help of G Power software [6]. and recalculation
was done by another author (k = 0.84). This software calculates post hoc
power on the basis of Cohen’s criteria of small, medium and large effect
size [7]. Studies whose power was found to be less than 80% were
considered underpowered.
51 studies were found to be inconclusive out of total
276. Out of these studies no study was found to be adequately powered
(power >80%) for small effect size. Only 8 (15.6%, 95% CI 8.1 to 28)
studies were adequately powered for medium effect size and 30 (58.8%, 95%
CI 45.1% to 71.2%) studies were adequately powered for large effect size.
22 (43.1%, 95% CI 30.5% to 56.7%) studies were under-powered even for
larger effect size.
Similar results were obtained in studies done for
negative studies published in western journals. In a study done by Keen,
et al. [2] it was observed that out of all negative studies
published in rheumatology literature in 2001-2002, only 50% studies had
adequate power [2]. In a similar study done by Freedman, et al. [8]
it was observed that out of 25 negative studies published in orthopedic
journals, no study had adequate power (80%) for small effect size
according to cohens criteria and only 48% studies had adequate power for
large effect size. Bedard, et al. [9] observed that out of 423
clinical trials only 45 (10.6%), 138 (32.6%) and 233 (55.1%) trials were
adequately powered for small, medium and large effect size, respectively
[9].
We can conclude that most of the inconclusive studies
published in Indian medical journals are underpowered to see the actual
difference between the groups; thus their validity is questionable.
References
1. Jaykaran, Yadav P, Chavda N, Kantharia ND. Some
issue related to the reporting of statistics in clinical trials published
in Indian medical journals: A Survey. International J Pharmacol.
2010;6:354-9.
2. Keen HI, Pile K, Hill CL. The prevalence of
under-powered randomized clinical trials in rheumatology. J Rheumatol.
2005;32:2083-8.
3. Freiman JA, Chalmers TC, Smith H, Kuebler RR. The
importance of beta, the type II error and sample size in the design and
interpretation of the randomized controlled trial: survey of 71 "negative"
trials. N Engl J Med. 1978;299:690-4.
4. Halpern S, Karlawish JH, Berlin JA. The continuing
unethical conduct of underpowered clinical trials. JAMA. 2002;288:358-62.
5. Altman DG. The scandal of poor medical research.BMJ.
1994;308:283-4.
6. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3:
A flexible statistical power analysis program for the social, behavioral,
and biomedical sciences. Behavior Research Methods. 2007;39:175-191.
7. Cohen J. Statistical power analysis for the
behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum; 1988.
8. Freedman KB, Back S, Bernstein J. Sample size and
statistical power of randomised, controlled trials in orthopaedics. J Bone
Joint Surg. 2001;83:397-402.
9. Bedard PL, Krzyzanowska MK, Pintilie M,
Tannock F. Statistical power of negative randomized controlled trials
presented at American Society for Clinical Oncology Annual Meetings. J
Clin Oncol. 2007;25:3482-7.
|
|
|
|