Various studies from India and other developing countries have validated the IMCI algorithm in both 7 days – 2 months and 2 months-5 years age groups [1-6]. However, there is little experience available on the validity of the algorithm after expanding its scope to include 0-7 days age group specifically. Subsequent to the formal development of the IMNCI algorithm which covers the complete 0-2 months age group, the present study was planned to validate it.

The study was conducted in the pediatrics unit of a medical college hospital in New Delhi, India during the period April 2005 to February 2006. The study was spread over 11 months duration to minimize the seasonal variation in morbidities. The subjects were enrolled as and when they came in contact with study team, in both the inpatient and outpatient department (OPD) so that illnesses of varying severity could be evaluated.

A total of 419 outborn young infants (0-2 months), who presented to OPD or emergency room of the treating unit for a fresh episode of illness formed the study group. These subjects were managed according to the protocol of treating unit under the supervision of the senior faculty. All relevant investigations were performed as indicated. As per the hospital policy, a birthweight of 1.5 kg or less (very low birth weight) by itself was a sole criterion for admission even if the baby was otherwise well. The decision of treating unit regarding diagnosis and treatment was considered as the ‘Gold Standard’.

For these cases, all the particulars and signs listed in IMNCI algorithm were recorded in the pre-designed proforma in the same order. The treatment steps were also identified according to IMNCI algorithm and recorded. These classification and treatment noted were not used for actual intervention or treatment. The actual diagnosis and therapy was determined by the admitting and treating unit. Parental consent for inclusion in the study and for follow up visit was taken in every case.

The study subjects were either admitted or sent home after initial evaluation, depending upon nature and severity of illness. Hospitalized cases were discharged on recovery while those kept under observation were sent home after demonstrating adequate response to administered therapy and/or establishment of a definite diagnosis which could be managed at home. The recruited infants were followed up to determine the outcome. For hospitalized subjects, this was restricted till discharge/death/leaving against medical advice while the outpatient recruits were followed up as per IMNCI recommendations and again after one week. Dietary therapy/advice was given to every child with low birth weight or those with feeding problem. Every unimmunized or incompletely immunized child was immunized.

The study subjects were divided into 0-7 days and 7 days-2 months age groups to study the feasibility and utility of IMNCI algorithm in the early neonatal period in particular. The data were then combined to study the utility of algorithm for <2 month old infants. The data were entered in Microsoft Excel data sheet and analysis was done using SPSS software version 10. A sample size of 120 neonates in each group was calculated to be sufficient to detect a difference of 10% in diagnostic agreement from the gold standard with 90% power and alpha of 0.05.

The efficacy of IMNCI algorithm to correctly identify sick young infants requiring referral was evaluated in terms of its sensitivity and specificity to identify cases who received in-patient treatment as per the gold standard. Further, broad diagnostic and therapeutic agreements between the gold standard and IMNCI were also compared. Broad diagnostic agreement between the two was categorized as no diagnostic mismatch, underdiagnosis, overdiagnosis and difference in diagnosis. The serious bacterial infections were not subdivided into sepsis, pneumonia, meningitis while analyzing the diagnostic agreement. ‘Underdiagnosis’ included cases where one or more illness recorded as per the gold standard was not covered by IMNCI and/or would not be referred using IMNCI algorithm though needed hospitalization. Similarly ‘Overdiagnosis’ included cases where morbidity recorded as per IMNCI was not confirmed by the gold standard and or those which would have been referred using IMNCI algorithm but did not need hospitalization. If there was a difference in diagnosis between ‘Gold Standard’ and IMNCI (e.g. hypocalcemic seizures vs. Possible Serious Bacterial Infection (PSBI), meconium aspiration syndrome vs. PSBI), it was considered as diagnostic mismatch. Standard statistical tests like Pearson’s test, chi square test, Fischer’s exact test, sensitivity, specificity, positive predictive value and the negative predictive value were used to analyze the results.

A total of 419 infants between 0-2 months who fulfilled the study criteria were investigated. Of these, 176 (42%) were 0-7 days and 243 (58%) were 7 days to 2 months of age. In 40 (9.5%) cases, either the follow up visit was not adhered to or the admitted patients left against medical advice. Since the admission diagnosis made by a trained pediatric resident and the basic investigation work up was available, these subjects, they have also been included for analysis.

Out of 419 patients, 124 (29.6%) were recruited from the outpatient department and 295 (70.4%) from the emergency room. In 0-7 days age group, 41 patients were taken from OPD and 135 patients from emergency room. Corresponding figures for 7 days – 2 months age group were 83 and 176, respectively. As per the management decided by the treating unit, 351 (84.2%) were hospitalized and 68 (15.8%) were sent back home and treated on outpatient basis after initial evaluation. Compared to patients recruited from OPD, emergency room recruits were significantly more likely to be hospitalized [33/41 (85.4%) vs 130/135 (96.3%) in 0-7 days group and 32/83 (38.6%) vs 156/160 (97.5%) in 7 days -2 months group; P value 0.021 and <0.001, respectively].

Out of 419 patients enrolled, 348 (83.1%) improved, 31 (7.4%) died and 40 (9.5%) patients were lost to follow up. The overall mortality in the 0-7 days group was significantly higher as compared to 7 days – 2 months group (12.5% vs 3.7%; P value 0.003).

The study cases frequently had co-existent morbidities as only about one-third of subjects had a single morbidity. Mean number of illness was 2.14, 2.04 and 2.08 in 0-7 days, 7 days – 2 months and 0-2 months, respectively. IMNCI algorithm in comparison to the gold standard, detected slightly lower proportion of co-existent morbidities (mean 1.88, 1.66 and 1.75 in the 0-7 days, 7 days – 2 months and 0-2 months, respectively). Infants requiring referral as per IMNCI algorithm had significantly greater co-existence of morbidities (mean 1.93 vs 1.4, 1.96 vs 0.80, 1.94 vs 0.93 in 0-7 days, 7 days -2 months and 0-2 months, respectively; P value 0.015, <0.001, <0.001 in the three age groups, respectively).

Table I details the morbidity profile observed as per the treating unit. Low birth weight requiring treatment (including counseling for feeding) contributed to multiplicity of illnesses in 124 (29.6%) infants. Thus, a significant proportion of co-existent morbidities were in association with low birth weight. Majority of the diagnosis (80%) made were either totally or partially covered by the algorithm. The sensitivity of algorithm to identify bacterial infection was 88.5% while the specificity was relatively low (57.4%).

Table I  Morbidity Profile [Number (%)] as per the “Gold Standard”

The sensitivity of IMNCI criteria in correctly identifying infants needing referral was 97%, 94% and 95%, and the specificity was 85%, 87% and 87% in 0-7 days, 7 days – 2 months and 0-2 months, respectively. The positive predictive value and negative predictive values for these age groups were 99%, 96%, 97% and 69%, 80%, 78%, respectively.

Table II presents the results of diagnostic agreement between IMNCI algorithm and the Gold Standard. Out of 80 cases with difference in diagnosis, 31 (38.7%) had birth asphyxia with hypoxic-ischaemic encephalopathy, 16 (20%) had hypocalcemic seizures, 11 (13.7%) had meconium aspiration syndrome, and 7 (8.8%) had hemorrhagic disease of newborn. Other conditions included respiratory distress syndrome (9 cases), transient tachypnea of newborn (4 cases), and neonatal seizures (2 cases). However, all of them were referred as PSBI. Among 78 cases with under-diagnosis, 20 cases of sepsis were missed by the IMNCI algorithm. Others had surgical conditions (17 cases), upper respiratory tract infection (14 cases), underestimation of severity of jaundice, congenital heart disease, septic arthritis, cephalhematoma, regurgitation of feeds and Down syndrome. Of 54 cases with overdiagnosis, 22 (40.7%) had breast fed stools 19 and (35.2%) were categorized as very low weight as per IMNCI algorithm, and would have been referred. As the protocol followed by the hospital considered admission only if the weight was less than 1.5 kg, IMNCI algorithm over diagnosed serious illness in some of the infants as it uses a different weight cutoff for referral/admission. Other over-diagnoses included overestimation of severity of dehydration and jaundice, upper respiratory tract infection (URI) being categorized as PSBI due to presence of fever.

TABLE II  Diagnostic Agreement Between ‘Gold Standard’ and IMNCI  Algorithm

A total of 131 infants assessed had jaundice. As per IMNCI algorithm, 61 of these were classified as having ‘jaundice’ and 70 as ‘severe jaundice’. The therapy decided in the hospital used serum bilirubin, age of the baby, gestation and other risk factors. Out of 61 infants classified as "jaundice" by IMNCI, 12 (20%) needed intervention (phototherapy alone in 10, phototherapy with exchange transfusion in 2). Of 70 infants classified as "severe jaundice’ by IMNCI, 8 had direct hyperbilirubinemia, 54 required treatment for jaundice, and no intervention was required in the remaining eight (11%). Thus the algorithm under-diagnosed the severity of jaundice in few subjects (12/131) and over-diagnosed (8/131) the severity in few subjects. Of the 76 cases identified as diarrhea by the algorithm, 22 (29%) had breast fed stools.

Assessment of feeding problems was done as per the IMNCI algorithm. About 35-40% of patients had some feeding problem and required counseling for the same. About one third young infants were not able to feed at all due to sickness or due to very low weight for age. Out of 66 cases treated on outpatient basis, 20 required feeding counseling and 35 required immunization counseling.

An important expectation of IMNCI algorithm for young infants is early recognition of severe morbidity for appropriate referral to a higher level of health facility. The study found that the IMNCI algorithm for young infants performed well in appropriately identifying cases for referral among both 0-7 days and 7 days-2 months age group. Majority (80%) of diagnoses made by the treating units were either totally or partially covered by the algorithm. There was complete agreement of diagnoses between IMNCI and the gold standard in about 50% of subjects. Complete diagnostic mismatch due to difference of diagnosis was present in 19% subjects. Since, majority of subjects with difference in diagnoses were appropriately referred. Thus, from a practical stand point, diagnostic mismatch due to difference in diagnosis did not affect algorithm’s performance.

Majority of cases with under-diagnosis had surgical conditions or URI. Patients having surgical conditions will require referral and most of them will probably be referred as IMNCI has a provision for assessment of other problems. URI was responsible for over-diagnosis and unnecessary referral as PSBI in some patients due to coexistent fever. The challenge will be to work around this common morbidity in the algorithm. Majority of patients with over-diagnosis had either breast fed stools or were categorized as very low weight by the algorithm. Though cases with breast fed stools will not be referred unnecessarily, they will get inappropriate treatment for diarrhea. IMNCI classifies neonates with birth weight less than 2.1 kg as very low weight and calls for referral of these cases. However, not all neonates less than 2.1 kg need urgent referral for admission. This could be another area for refinement of algorithm.

We did not find other published studies to compare our findings as the inclusion of 0-7 days period in IMCI algorithm is unique to our country. Prior to this study, there is only one report available from India assessing possibility of covering 0-7 days age group with generic IMCI algorithm. This study concluded that the performance of WHO IMCI algorithm is within an ‘acceptable range’ for both 0-7 days and 7 days – 2 months. However, sensitivity of algorithm can be further increased if yellowness of lower extremities/palms/soles is included in the algorithm [3]. The new adapted IMNCI algorithm includes jaundice and yellowness of palms and soles. The sensitivity and specificity of IMNCI algorithm in our study were better as compared to previous study [3], most likely as a result of subsequent modifications done in generic IMCI algorithm for covering the early neonatal period (0-7 days) in India [5].

Although, in our study, the ‘Gold Standard’ diagnoses were made by pediatric faculty, the researcher filling the IMNCI proforma was not always blinded to these. Thus, a possibility of bias cannot be ruled out. Further, the study was done in a tertiary health care facility catering to patients who are sicker and also more patients were recruited from emergency room. The usefulness of the algorithm in the community setting may be altered due to a variable and different mix of sick babies. The present study quantified the utility of IMNCI algorithm on the basis of assessment undertaken by a pediatric resident and this is likely to be affected in the hands of peripheral health workers.

To conclude, the IMNCI approach in young infants has good sensitivity and specificity for referring children with severe illness along with provision for preventive services of immunization and feeding counseling. The diagnostic mismatch observed highlights the need for having a different strategy for the management of sick young infants in the facility once they have been identified and referred using IMNCI algorithm. The efforts being made to strengthen the facility based management of sick young infants is an important step in this direction; the benefits of which are still to be systematically studied.

Contributors: SK: acquisition of data, analysis, interpretation and drafting of manuscript; VS: concept and design, analysis, interpretation, revision and final approval; AKD: concept and design, final approval; JC: design and drafting of manuscript. VS shall act as guarantor.

Funding: None.

Competing interests: None stated.