Home            Past Issues            About IP            About IAP           Author Information            Subscription            Advertisement              Search  

   
research letter

Indian Pediatr 2011;48: 490-491

Negative Studies Published in Indian Medical Journals are Underpowered


Jaykaran, Deepak Saxena*, Preeti Yadav and NK Kantharia,

Department of Pharmacology and Community Medicine,* Government Medical College, Surat, Gujarat, India.

Email: [email protected] 
 


51 inconclusive studies published in 14 Indian medical journals were analyzed for adequacy of power by post hoc power calculation. No study was found to be adequately powered (>80%) at small effect size, only 8 studies were adequately powered for medium effect size and 30 studies were adequately powered at large effect size.


Inconclusive studies published in medical journals should have proper reporting of power and sample size so that their validity can be assessed by readers. A study may become inconclusive because of two main reasons one there may be actually no difference between the groups for the outcomes or sample size/power of study is not adequate to measure that difference. Results of a study can only be considered valid when the study is adequately powered to see the difference between the groups [1]. It has been observed in the studies done for articles published in westerns medical journals that many of the studies including inconclusive and negative studies are not adequately powered to detect difference between the groups [2,3]. Publications of these underpowered studies are considered as unethical because they exposes the patients to adverse effects of interventions but unable to detect which intervention is superior [4,5]. Though many studies are published for assessment of articles published in western medical journals for adequacy of power, similar data are lacking for the Indian Medical Journals. So this study was designed with the aim of critical evaluation of inconclusive studies published in 14 Indian medical journals for adequacy of power by post hoc power calculation.

All the Pubmed indexed journals subscribed by central library of our institute were taken into consideration. These journals were Indian Pediatrics (IP)(inconclusive studies 6/13), Annals of Indian Academy of Neurology (AIAN)(5/13), Indian Journal of Orthopedics (IJ Ortho) (inconclusive studies 7/63), Indian Journal of Critical Care Medicine (IJCCM) (inconclusive studies 2/15), Indian Journal of Dermatology, Vanereology and Leprology (IJDVL) (inconclusive studies 4/16), Indian Journal of Nephrology (IJN) (inconclusive studies 1/14), Indian Journal of Dermatology (IJD) (inconclusive studies 3/21), Indian Journal of Ophthalmology (IJO) (inconclusive studies 1/15), Indian Journal of Urology (IJU) (inconclusive studies 8/24), Indian Journal of Anesthesia (IJA) (inconclusive studies 13/46), Indian Journal of Psychiatry (IJ Psy) (inconclusive studies 1/8), Indian Journal of Medical Research (IJMR) (inconclusive studies 0/12), Indian Journal of Medical Science (IJMS) (inconclusive studies 0/6) and Indian Journal of Community Medicine (IJCM) (inconclusive studies 0/10). All the original articles published in 2009 from these journals were evaluated to identify inconclusive studies. Studies were considered inconclusive studies on the basis of these criteria-one: primary outcome is not statistically significant, second: most important outcome is not statistically significant (if primary out come is not reported). Post hoc power of all these inconclusive studies were calculated with the help of G Power software [6]. and recalculation was done by another author (k = 0.84). This software calculates post hoc power on the basis of Cohen’s criteria of small, medium and large effect size [7]. Studies whose power was found to be less than 80% were considered underpowered.

51 studies were found to be inconclusive out of total 276. Out of these studies no study was found to be adequately powered (power >80%) for small effect size. Only 8 (15.6%, 95% CI 8.1 to 28) studies were adequately powered for medium effect size and 30 (58.8%, 95% CI 45.1% to 71.2%) studies were adequately powered for large effect size. 22 (43.1%, 95% CI 30.5% to 56.7%) studies were under-powered even for larger effect size.

Similar results were obtained in studies done for negative studies published in western journals. In a study done by Keen, et al. [2] it was observed that out of all negative studies published in rheumatology literature in 2001-2002, only 50% studies had adequate power [2]. In a similar study done by Freedman, et al. [8] it was observed that out of 25 negative studies published in orthopedic journals, no study had adequate power (80%) for small effect size according to cohens criteria and only 48% studies had adequate power for large effect size. Bedard, et al. [9] observed that out of 423 clinical trials only 45 (10.6%), 138 (32.6%) and 233 (55.1%) trials were adequately powered for small, medium and large effect size, respectively [9].

We can conclude that most of the inconclusive studies published in Indian medical journals are underpowered to see the actual difference between the groups; thus their validity is questionable.

References

1. Jaykaran, Yadav P, Chavda N, Kantharia ND. Some issue related to the reporting of statistics in clinical trials published in Indian medical journals: A Survey. International J Pharmacol. 2010;6:354-9.

2. Keen HI, Pile K, Hill CL. The prevalence of under-powered randomized clinical trials in rheumatology. J Rheumatol. 2005;32:2083-8.

3. Freiman JA, Chalmers TC, Smith H, Kuebler RR. The importance of beta, the type II error and sample size in the design and interpretation of the randomized controlled trial: survey of 71 "negative" trials. N Engl J Med. 1978;299:690-4.

4. Halpern S, Karlawish JH, Berlin JA. The continuing unethical conduct of underpowered clinical trials. JAMA. 2002;288:358-62.

5. Altman DG. The scandal of poor medical research.BMJ. 1994;308:283-4.

6. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods. 2007;39:175-191.

7. Cohen J. Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum; 1988.

8. Freedman KB, Back S, Bernstein J. Sample size and statistical power of randomised, controlled trials in orthopaedics. J Bone Joint Surg. 2001;83:397-402.

9. Bedard PL, Krzyzanowska MK, Pintilie M, Tannock F. Statistical power of negative randomized controlled trials presented at American Society for Clinical Oncology Annual Meetings. J Clin Oncol. 2007;25:3482-7.
 

 

Copyright© 1999 by the Indian Pediatrics (Disclaimer)