The best answer to this question, of which of the two alternative interventions is better, can be studied in a double blind Randomized Control Trial (RCT) comparing the new drug (nebulized hypertonic saline) against standard therapy (nebulized epinephrine). However, RCT’s have inherent problems, especially in the context of trials in children. According to Mc Culloch and colleagues, RCTs involve difficult blinding, require large samples, long duration, and are very expensive [7]. Others have noted that it is difficult to recruit cases [8].

Drug trials often include a provision of interim analysis and stopping of the trial if large differences between treatment groups are detected. We did this study to see if instead of such interim analyses, Cumulative Sum (CUSUM) can be used for continuous monitoring of the use of the study drug.

Cumulative Sum (CUSUM) is a statistical technique used in industry for quality control. Positive weights are given for successes and negative weights for failures such that, as the process continues, the cumulative score stays close to the zero line. Control lines are drawn so that if there are more failures or success than would be expected by chance, the lines would be crossed. These lines can be drawn using boot-strapping methods, as the sequence of failures and success depends on chance. By repeatedly reordering the same data randomly in say 10,000 iterations, the limits of the CUSUM that can occur by chance can be defined [9]. The highest CUSUM and the lowest CUSUM is noted for each iteration. The upper limit of CUSUM is considered the mean upper CUSUM +2 SD and the lower limit is considered as the mean lower CUSUM-2SD.

CUSUM has been used to study antimicrobial treatment in neutropenic patients [10] and in the qualitative assessment of clinical competence [11]. Watkins and colleagues used CUSUM for the detecting Ross River Virus disease [12]. We have used this novel statistical tool to test if hypertonic saline was better or worse than the standard therapy with epinephrine. In this communication we report the procedure, the software we developed and the results of the study.

Children enrolled in the second half of the bronchiolitis season (the next 3 months after the initial 35 patients were recruited to receive standard therapy) received nebulized hypertonic (3%) saline, 3 mL every 6 hours. Clinical score and heart rate were monitored. Success and failures were measured by the same criteria as with epinephrine. The CUSUM score with nebulized hypertonic saline was plotted. It was decided a priori that if failure rate crossed the lower CUSUM control line, the trial would be stopped immediately. If however, successes were more and it crossed the upper control line, the study would be continued till the end of bronchiolitis season and it would be clear that new drug is superior to standard therapy. In the absence of suitable software, we developed custom-built open-source software [15] which allows inputting of any initial series data (epinephrine in the study). The software does the boot-strapping and calculates the limits for the control line and the stopping rule.

Conventional statistical methods were employed to examine differences between groups. To compare means and the confidence intervals, we used statistical software (Confidence Interval Analysis) (CIA) [16]. This study was approved by the Hospital Ethics Committee.

64 patients were enrolled in the study; 35 received nebulized epinephrine and subsequently, 29 received nebulized hypertonic saline. Details of children in the two groups are given in Table I. There was no statistically significant difference in the age, sex distribution, clinical score and heart rate at admission. There was no significant difference in the duration of stay between the groups.

TABLE I  Population Characteristics of The Two Groups

With the initial trial with epinephrine, there were 5 children with ‘treatment failure’ (2 with worsening of clinical score and 3 with tachycardia). The resultant weightage for success and failure were +0.286 and –1.714, respectively. The highest CUSUM during the trial using epinephrine was +5.429 and the lowest CUSUM was –0.571. Boot-strapping was done with 10,000 iterations. Increasing iterations beyond this number did not significantly alter the CUSUM maximum and minimum values. The maximum CUSUM was +2.253 (SD 1.342) and minimum was –2.259 (SD 1.337). 2 SD (standard deviation) was added to the maximum CUSUM, and 2 SD was subtracted from the minimum CUSUM values, to draw the control lines as shown in Fig. 1. The CUSUM score while using hypertonic saline stayed above the zero.

Fig. 1 Real time CUSUM plot with nebulized hypertonic saline with CUSUM control lines.

Many clinical trials include some strategy for interim analysis and early stopping, if large differences between treatment groups are detected. This design feature can reduce the study participants’ exposure to inferior treatment in addition to saving time and resources. Interim analyses are done at different predetermined points during the study. Instead of using the traditional form of interim analysis, we used the CUSUM to continuously monitor the new drug (nebulized hypertonic saline). In this ‘proof-of-concept study’ we monitored the new drug with a view to terminate the study if the drug was less useful than standard therapy with epinephrine in preventing deterioration in bronchiolitis. Other than using CUSUM for this stopping rule, we employed conventional statistical methods to compare the two treatment groups. We found that CUSUM monitoring is possible in the context of clinical trials.

The software we have developed allows CUSUM to be deployed easily in a number of clinical contexts besides drug trials. The use of this software in evaluating the competence of ophthalmic surgeons for cataract surgery has been published [17].

It must be emphasized that epinephrine is not used as standard therapy, in the treatment of bronchiolitis universally. In our proof-of-concept study we used epinephrine as ‘standard therapy’ for the first 3 months to act as controls, and study drug in the second half of the bronchiolitis season. We did not determine the sample size for this study as we felt that the stopping rule would govern numbers of patients recruited for the trial drug. However, we should have used conventional methods to calculate sample size to determine the number of children receiving standard therapy. The study drug trial recruitment could also be continued till the required sample is reached, unless the study has to be curtailed early for breeching the stopping rule. Another major limitation of the study was the comparison of two interventions at different time periods in a sequence. Changes other than the drug might have influenced the results during two different time periods.

This novel technique of using CUSUM helps to stop trials early, if the study drug performs worse than the comparator. This will help limit the risks to study subjects. The parameters to be monitored using CUSUM could be side effects or therapeutic benefits. It can be used for comparisons against placebo or against standard therapy. The new tool can be used within open label RCT to allow for continuous monitoring of the trial. The idea for a clinical CUSUM calculator was taken from industry (quality control mechanism) to provide a simple method to monitor a drug trial. Paradoxically, industry itself may find the simple software we developed, handy for quality control.

Compared to RCT this tool, however, does not allow blinding and randomization easily. Though use of CUSUM provides the advantage of continuous monitoring of the trial, it cannot replace the standard double blind RCTs. We believe that the software we developed for the study allows easy boot-strapping and drawing of control lines and thus has potential for use in many clinical situations.

Contributors: NG, AP, AM and JP conceived the project, NG conducted the clinical trial. AP helped NG and JP in the statistical analysis and development of software.NG, AM and JP were responsible for the write up. JP will act as guarantor.

Funding: None; Competing interests: None stated.

Note: The software used in this study was custom built for the study. It can be downloaded free from the internet.

1. Mandelberg A, Tal G, Witzling M, Someck E, Houri S, Balin A, et al. Nebulized 3% hypertonic saline solution treatment in hospitalized infants with viral bronchiolitis. Chest. 2003;123:481-7.

2. Tal G, Cesar K, Oron A, Houri S, Ballin A, Mandelberg A. Hypertonic saline/epinephrine treatment in hospitalized infants with viral bronchiolitis reduces hospitalization stay: 2 years experience. Isr Med Assoc J. 2006;8:169-73.

3. Wark PA, McDonald V. Nebulized hypertonic saline for cystic fibrosis. Cochrane Database Syst Rev. 2000;2:CD001506.

4. Mandelberg A, Amirav I. Hypertonic saline or high volume normal saline for viral bronchiolitis: mechanisms and rationale. Pediatr Pulmonol. 2010;451:36-40.

5. Zhang L, Mendoza-Sassi RA, Wainwright C, Klassen TP. Nebulized hypertonic saline solution for acute bronchiolitis in infants. Cochrane Database Syst Rev. 2008;4:CD006458.

6. Mathew JL. What works in bronchiolitis? Indian Pediatr. 2009;46:154-8.

7. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomised trials in surgery: problems and possible solutions. BMJ. 2002;324:1448-51.

8. Yeung V. Clinical trials in children. In: Pediatric Drug Handling. Available from http://www.pharmpress.com/shop/samples/paeditaric_sample_chapter.pdf. Accessed October 12, 2009.

9. Taylor WA. Change-Point Analysis: A Powerful New Tool for Detecting Changes. Available from: URL http://www.variation.com/cpa/tech/changepoint.html. Accessed October 11, 2009.

10. Kinsey SE, Giles FJ, Holton J. Cusum plotting of temperature charts for assessing antimicrobial treatment in neutropenic patients. BMJ. 1989;299:775-6.

11. Ravn LI, Sprehn M, Pedersen CB. The Cusum score. A tool for evaluation of clinical competence. Ugeskr Laeger. 2001;163:3644-8.

12. Watkins RE, Eagleson S, Veenendaal B, Wright G, Plant AJ. Applying cusum-based methods for the detection of outbreaks of Ross River virus disease in Western Australia. BMC Med Inform Decis Mak. 2008;8:37.

13. Uyan AP, Ozyurek H, Keskin M, Afsar Y, Yilmaz E. Comparison of two different bronchodilators in the treatment of acute bronchiolitis. Internet Journal of Pediatrics and Neonatology. 2003;3;1.

14. Beck R, Elias N, Shoval S, Tov N, Talmon G, Godfrey S, et al. Computerized acoustic assessment of treatment efficacy of nebulized epinephrine and albuterol in RSV bronchiolitis. BMC Pediatr. 2007;7:22.

15. Foresee: CUSUM Calculator for Clinical Control (4-C Software). Available from http://jacob.puliyel.com/calculator/. Accessed August 7, 2011.

16. Confidence Interval Analysis (CIA) Software. Available from http://www.som.soton.ac.uk/research/sites/cia/. Accessed August 7, 2011.

17. Puliyel A, Puliyel J. CUSUM for monitoring competency: computer software is useful for bootstrapping and real time CUSUM plotting. Br J Ophthalmol. 2011;95:295-6.

18. Kuzik BA, Al-Qadhi SA, Kent S, Flavin MP, Hopman W, Hotte S, et al. Nebulized hypertonic saline in the treatment of viral bronchiolitis in infants. J Pediatr. 2007;151:266-70.