This was a single center, retrospective diagnostic test evaluation. Inclusion criteria were: preterm infants (≤33 weeks gestation) with birth-weight <1500 g, who were born between January 2006 and December 2008 and treated in the neonatal intensive care unit (NICU) of Bikur Holim hospital in Jerusalem, Israel. During this period, 13,391 babies were born in the hospital, out of whom 173 weighed less than 1500 g. Out of 45 patients with definitive diagnosis of sepsis based on positive blood cultures (proven sepsis), 25 neonates were randomly selected. Similarly, out of the remaining 128 cases without any clinical or microbiological evidence of sepsis, 25 were randomly selected. Four of these patients were eventually excluded from the study due to early sepsis (1 patient) or gestational age >33 weeks (3 patients).

The study was approved by an Institutional Review Board and all patients’ parents gave their written informed consent.

RALIS is a computerized mathematical algorithm for continuous monitoring of patients in order to detect the potential onset of sepsis. End users are medical personnel, who enter the relevant data, into the algorithm concomitant to the routine medical documentation. The algorithm was originally designed based on a cohort of 200 subjects, 100 with definite diagnosis of LOS and 100 who were obviously healthy. This is the first study of a controlled diagnostic evaluation of this tool. The algorithm was configured to process measurement values for a plurality of vital signs and to generate an alarm signal indicative of the onset of sepsis. The analyzed vital signs included heart rate, respiratory rate, core body temperature, body weight, documented desaturations (<85%), and documented bradycardias (<100 beats per minute). For each vital sign one or more threshold values were pre-defined. For example, for the temperature two threshold values were defined – a high alarm limit of 38.3°C and a low alarm limit of 35.5°C. If the temperature exceeded the first threshold value, or if it was less than the second threshold value, a conditional warning signal was asserted. The presence of desaturation or bradycardia events during the two hour interval was marked with a plus (+) sign, while the absence of such events was marked with a minus (–) sign. There was no significance to the absolute number of events during an interval. All parameters were monitored 12 times a day, except for body weight, which was determined once daily. When using the algorithm, a 48-hour training period was given for patient-tailored calibration in order to determine the personal baseline, to which all deviations were compared. Final readout was given in a 0-10 relative scale, with baseline level defined as 0, and the threshold for the definition of sepsis defined as 5. The sepsis factor (S-factor) was then plotted against time (Web Fig. 1).

All the clinical parameters described above were retrieved retrospectively and entered into the RALIS Sepsis Detecting System by two operators blinded to the purpose of the study. For each baby, we collected data every 2 hours for 10 consecutive days during their hospitalization (including 48-hour training period for patient-tailored calibration). Data collection starting point for the ‘proven sepsis’ group was 10 days prior to the clinical diagnosis or suspicion of sepsis made by the medical staff, according to the daily medical record. Data collection starting point for the ‘no sepsis’ group was at the age of 1-10 days, based on previous data on the average day (and range) of clinical suspicion of most LOS in this NICU (48-hour training period plus 7-8 days previously).

Statistical analysis: Statistical analysis was performed using SigmaStat, version 2.03 (Chicago, IL) and Minitab, Version 12.23 (State College, PA) softwares. Statistical significance was set at 0.05 levels.

The study population included 24 patients with ‘proven sepsis’ and 22 patients with ‘no sepsis’ (Table I).

TABLE I  Characteristics of Study Population

The algorithm positively identified 23 out of 24 cases of sepsis. Out of the 23 positively identified cases, only one case was identified clinically before RALIS, and in another case both modalities identified sepsis roughly at the same time. Thus, 21 cases (87.5%) could have benefited from an earlier diagnosis by RALIS (P<0.001) as compared to clinical diagnosis (Fig. 1). The algorithm identified sepsis 2.4 ± 2.0 days earlier (median 2.0 days). Subgroup analysis according to gestational age (25-28 vs. 29-33 weeks) or birthweight (≤1000 vs. >1000g) revealed no significant differences (Mann-Whitney Rank Sum Test on medians: 2.0 days in all subgroups).

The difference column denotes the difference between the average days of sepsis diagnosis by the clinician and RALIS™. Boxes show interquartile ranges, the small squares signs denote mean values, the horizontal lines denote median values, I bars represent the range of mean ± SD. The X signs represent highest and lowest observed values (range). Fig. 1 RALIS day of diagnosis compared to clinical diagnosis in proven sepsis group (n=24).

Only one case of definitive sepsis was missed by the algorithm (false negative rate of 4.2%). The overall sensitivity of RALIS was 95.8% (95%, CI 78.8-99.3%). The specificity, positive predictive value and negative predictive value of RALIS were 77.3% (95% CI 54.6-92.1%), 82.1% (95% CI 63.1-93.9%), and 94.4% (95% CI: 72.6-99.1%), respectively. The positive and negative likelihood ratios were 4.2 (95% CI 1.9-9.2) and 0.05 (95% CI 0.01-0.4), respectively. Subgroup analysis according to gestational age, birth-weight or gender revealed no significant differences in TN, FP, FN and TP between the subgroups.

The odds ratio (OR) for positive RALIS identification when sepsis was present was 78.2 (95% CI 8.3-732.1) (P<0.001) using univariate analysis. Multivariate analysis was done with sepsis diagnosis (proven vs. none), gender (males vs. females) and gestational age (wks) and/or birth-weight (g) as independent variables. The models that were used included logistic regression and linear regressions (general linear model, forward and backward stepwise regressions and best-fit model). In all models the only variable that was significantly associated with RALIS positive identification was proven sepsis (P<0.001), the OR was 70.2 (95% CI 6.3-777.1).

The number of days RALIS identified sepsis before clinical diagnosis according to the causative organism Coagulase Negative Staphylococcus (CONS) (n=10, 41.7%): 3.3±2.4 days; Klebsiella (n=7, 29.2%): 2.2±1.0 days; Enterococcus (n=3, 12.5%): 1.0±2.0 days; E.Coli (n=2, 8.3%): 1.0±1.4 days; and S. Aureus (n=2, 8.3%, only one was diagnosed by RALIS): 1 day (P=0.197).

RALIS exhibited a high sensitivity and reasonable specificity to detect LOS in VLBW preterm infants. It correctly identified sepsis two days before the clinician. The high negative predictive value makes it a possible good screening test to exclude LOS.

The major limitation of this study was its retrospective design. The diagnostic ability of the algorithm was compared with day-to-day clinical decision making. It could thus be argued that even a clinician can make early diagnosis of sepsis if assessment of theses clinical signs (e.g. unexplained tachycardia or apnea) is made systematically without the algorithm. Final confirmation of this hypothesis can be made only in a prospective study. Other study limitations include the relatively small sample size, and the selection of patients that was performed only within the groups with or without culture proven sepsis. Clinical sepsis was not addressed by this study. In addition, the sepsis positive and negative groups in the current study significantly differed in mean gestational age and mean birth-weight. However, multivariate analysis showed that the ability of the test to positively diagnose sepsis was significantly associated only to the presence of proven sepsis.

Although many biological markers and cytokines that correlate with sepsis have been discovered over the years [1,8-12], most of these cannot be used as a single marker for early reliable detection of sepsis. Probably only integration of multiple parameters and markers could have the capability to predict the development of sepsis [13,14]. Recent systemic algorithm-based networks have been set for the analysis of systemic inflammation in humans [15]. However, despite some potentially promising results, most of these tests are still taken only based on clinical suspicion. The possibility of continuous monitoring of a set of serum highly sophisticated biomarkers does not provide a simple, cost-effective solution. Although having a high negative predictive value, very high FP rate of RALIS might lead to the overuse of unnecessary antibiotics in almost one in four infants, or at least subject these infants to sepsis work-up with invasive tests to try and exclude LOS. With increasing antibiotic resistance and awareness to reducing pain in preterm infants, this may be an issue of concern. Larger prospective studies should be done to confirm the ability of this algorithm to detect LOS in most VLBW preterm infants early enough, even before clinical suspicion.

Contributors: IG, GM, AE: conceived and designed the study and revised the manuscript for important intellectual content. IG will act as guarantor of the study; IG,YN,IV: collected data and drafted the paper; AR: analyzed the data and helped IG in manuscript writing. The final manuscript was approved by all authors. Funding: None; Competing interests: None stated.

1. Camacho-Gonzalez A, Spearman PW, Stoll BJ. Neonatal infectious diseases: evaluation of neonatal sepsis. Pediatric Clin North Am. 2013;60:367-89.

2. Makhoul IR, Sujov P, Smolkin T, Lusky A, Reichman B. Epidemiological, clinical, and microbiological characteristics of late-onset sepsis among very low birth weight infants in Israel: A national survey. Pediatrics. 2002;109:34-9.

3. Stoll BJ, Hansen N, Fanaroff AA, Wright LL, Carlo WA, Ehrenkranz RA, et al. Late-onset sepsis in very low birth weight neonates: the experience of the NICHD Neonatal Research Network. Pediatrics. 2002;110:285-91.

4. Vergnano S, Menson E, Kennea N, Embleton N, Russell AB, Watts T, et al. Neonatal infections in England: the NeonIN surveillance network. Arch Dis Child Fetal Neonatal Ed. 2011;96:F9-14.

5. Stoll BJ, Hansen NI, Bell EF, Shankaran S, Laptook AR, Walsh MC, et al. Neonatal outcomes of extremely preterm infants from the NICHD Neonatal Research Network. Pediatrics. 2010;126:443-56.

6. Murphy K, Weiner J. Use of leukocyte counts in evaluation of early-onset neonatal sepsis. Pediatr Infect Dis J. 2012;31:16-9.

7. Hengst JM. The role of C-reactive protein in the evaluation and management of infants with suspected sepsis. Adv Neonatal Care. 2003;3:3-13.

8. Chiesa C, Pellegrini G, Panero A, Osborn JF, Signore F, Assumma M, et al. C-reactive protein, interleukin-6, and procalcitonin in the immediate postnatal period: influence of illness severity, risk status, antenatal and perinatal complications, and infection. Clin Chem. 2003;49:60-8.

9. Auriti C, Fiscarelli E, Ronchetti MP, Argentieri M, Marrocco G, Quondamcarlo A, et al. Procalcitonin in detecting neonatal nosocomial sepsis. Arch Dis Child Fetal Neonatal Ed. 2012;97:F368-70.

10. Vouloumanou EK, Plessa E, Karageorgopoulos DE, Mantadakis E, Falagas ME. Serum procalcitonin as a diagnostic marker for neonatal sepsis: a systematic review and meta-analysis. Intensive Care Med. 2011;37:747-62.

11. Ng PC, Li K, Wong RP, Chui KM, Wong E, Fok TF. Neutrophil CD64 expression: a sensitive diagnostic marker for late-onset nosocomial infection in very low birthweight infants. Pediatr Res. 2002;51:296-303.

12. Ng PC, Li K, Wong RP, Chui K, Wong E, Li G, et al. Proinflammatory and anti-inflammatory cytokine responses in preterm infants with systemic infections. Arch Dis Child Fetal Neonatal Ed. 2003;88:F209-13.

13. Kabir K, Keller H, Grass G, Minor T, Stueber F, Schroeder S, et al. Cytokines and chemokines in serum and urine as early predictors to identify septic patients on intensive care unit. Int J Mol Med. 2003;12:565-70.

14. Toh CH, Ticknor LO, Downey C, Giles AR, Paton RC, Wenstone R. Early identification of sepsis and mortality risks through simple, rapid clot-waveform analysis. Implications of lipoprotein-complexed C reactive protein formation. Intensive Care Med. 2003;29:55-61.