From the Department of Pediatrics, Maulana Azad Medical College, New Delhi 110 002, India.
Reprint requests: Dr. H.P.S. Sachdev, Professor and Incharge, Division of Clinical Epidemiology, Department of Pediatrics, Maulana Azad Medical College, New Delhi 110 002, India. E-mail: [email protected]

Manuscript received: May 31, 1999; Initial review completed: July 15, 1999; Revision accepted: August 23, 1999.

Objective: To evaluate the utility of the WHO/UNICEF algorithm for integrated management of childhood illness (IMCI) between the ages of 1 week to 2 months. Design: Prospective observational. Setting: The Outpatient Department and Emergency Room of a medical college hospital. Methods: 129 infants presenting to Outpatient Department (n = 70) or Emergency Room (n = 59) were assessed and classified as per ‘IMCI’ algorithm and treatment required was identified. A detailed evaluation with all relevant investigations was also done for these subjects. The final diagnoses made and therapies instituted on this basis served as ‘gold standard’. The diagnostic and therapeutic agreement between the ‘gold standard’ and the ‘IMCI’ was computed. Results: More than one illness was present in 97 (75.2%) of subjets as per ‘gold standard’ (mean 2.1). Subjets having any referral criteria as per ‘IMCI’ algorithm had a greater (p = 0.002) co-existence of illnesses (mean 2.3 vs. 1.8 illnesses per child, respectively). IMCI algorithm covered majority (81-84%) of the recorded diagnoses either partly (40-41%) or fully (40-44%). The referral criteria proved quite sensitive (86-87%) in predicting hospitalization but had a lower specificity (53-58%). A total agreement with IMCI was found in 60-66% cases. The mismatch (34-40%) was more commonly of an overdiagnosis (21-23%) rather than underdiagnosis (15-21%). The sensitivity of the algorithm to identify serious bacterial infection was high (96.1-96.5%) while the specificity was relatively low (51.8-59.7%). Upper respiratory infection (URI) emerged as an important cause resulting in unnecessary referrals (13 out of 21 cases). Of the 43 cases identified as diarrhea by the algorithm, 6 had breast fed stools, which do not require any therapy. The ‘IMCI’ algorithm had a provision for preventive services of immunization and breastfeeding counseling (18% possibility of availing missed opportunities in both). Conclusion: There is a sound scientific basis for adopting IMCI approach even in young infants as co-existence of morbidities is frequent and severe illnesses are assessed with good sensitivity. However, there is a need to improve the specificity of referral criteria. Two important conditions identified for possible refinement are URI and breast fed stools.

Key words: Infant mortality, Integrated management of childhood illness, Neonatal mortality.

Infant mortality remains unacceptably high in developing countries, with about 8 million deaths occurring annually in infants, 5 million during the neonatal period(1). Overall for India, infant mortality comprises nearly 70% of under-five mortality(2). The neonatal component of infant mortality is fairly high ranging between 60% to 77% for all but four states(2). It is obvious that programs to reduce under five and infant mortality must now urgently address neonates and young infants.

Formulation of simplified programmatic guidelines for the management of sick young infants offers a formidable challenge since the clinical presentation of diseases in young infants is often non-specific; feeding difficulty, lethargy, irritability and respiratory distress are common denominators of a group of diseases, some of which may be of serious nature. In an attempt to overcome this limitation and to improve health-workers’ performance in managing sick young infants, World Health Organization (WHO) and United Nations Children’s Fund (UNICEF) have devised a separate algorithm for age group of one week up to two months in the Integrated Management of Childhood Illness (IMCI) approach(3,4). India is currently in the process of introducing the IMCI strategy. However, before it’s wide-spread implementation, the generic ‘IMCI’ algorithms require careful adaptation to reflect the epidemiological and cultural characteristics of the country.

The published experience for the ‘IMCI’ algorithms in India partains to older children(5). Further, there is scarce quantification of the upper range of expectations from this approach in young infants, namely the agreement between the ‘gold standard’ and a pediatric resident following the algorithm. The present study was therefore designed to generate relevant informa-tion in this context for the proposed algorithm for children between the ages of one week up to two months.

The study was performed in the Outpatient Department and Emergency Room between May 1997 to February 1998. Both Outpatient Department and Emergency Room settings were utilized so that illnesses of various types and severity could be evaluated. The study period was deliberately extended to ten months to minimize the effect of seasonal clustering of common morbidities. All subjects aged between 1 week to 2 months, who presented to the Outpatient Department or Emergency Room of the hospital for a fresh episode of an illness, were eligible for enrolment in the study. The recruited subjects were selected from the eligible cases in a randomized manner.

For the children recruited in the study, the WHO/UNICEF algorithm for ‘Integrated Management of Childhood Illness’ was referred to. Every study infant was assessed and classified according to these guidelines, treatment steps identified and information was recorded in a proforma. A pediatric post-graduate trainee (RG) performed this assess-ment during the second year of the residency program. The study subjects were then assessed, examined and managed according to the proto-col of the treating unit under the supervision of faculty and/or pediatric senior residents. All relevant investigations (including blood counts, chest radiograph, stool examination, blood cultures, lumbar puncture, etc.) were performed on the basis of history and detailed clinical examination. Based on this detailed clinical evaluation and relevant investigations, final diagnoses were made and therapies instituted. These diagnoses and treatments were con-sidered as the ‘gold standard’.

The study children were either admitted or sent home after initial evaluation, depending upon the nature and severity of illness. The hospitalized subjects were followed up till discharge or death. Other children were called for follow up one week later to determine the final outcome. Each unimmunized or incompletely immunized sick child was immunized and dietary therapy/advice was given to every child with low birth weight or one with a feeding problem.

Two categories of possible diagnoses and treatments were therefore available for each recruited study subject, namely, ‘gold standard’ and ‘IMCI algorithm’. A sample size of 117 was calculated to be sufficient to detect a difference of 10% in diagnostic agreement from the gold standard with 90% power and an alpha of 0.05. The data was entered and analyzed with the Epi-Info version 5.00 software. The diagnostic and therapeutic agreements between the ‘gold standard’ and the ‘IMCI’ algorithm were computed. The utility of referral criteria was also evaluated. Standard statistical tests performed included Student’s ‘t’ test, Chi-square test, Fischer’s exact test, Odds Ratio, sensitivity, specificity, positive predictive value and negative predictive value.

A total of 129 infants (70 from Outpatient Department and 59 from Emergency Room) were evaluated, out of which 76 (59%) were hospitalized and 53 (41%) were sent back on outpatient basis after initial evaluation. Of the 129 recruited subjects, 9 (7%) expired and 96 (74.4%) improved. In 24 cases either follow-up visit was not adhered to or the admitted patients left the hospital against medial advice. The diagnoses of these 24 cases could from one aspect be considered uncertain since the total follow-up was not available. Thus out of 129 infants, 105 cases (81%) were followed up as per the protocol. Of these, 55 (52.4%) and 50 (47.6%) were emergency and OPD cases, respectively. As per the ‘gold standard’ manage-ment 69 (65.7%) were hospitalized and 36 (34.3%) were sent back after evaluation. The subsequent data analysis will be presented as two sets, namely for all the 129 subjects and for only 105 cases in which the follow-up was certain.

Three fourths (75%) of the children had two or more co-existent morbidities as per the gold standard diagnoses. In comparison to the ‘gold standard’, the ‘IMCI’ module documented a slightly lower number of co-existent morbidities (mean 2.1 vs 1.8, respectively, p = 0.001). Infants requiring referral as per ‘IMCI’ algorithm had significantly greater co-existence of morbidities (2.3 ± 0.8 vs 1.8 ± 0.8, p = 0.002) thereby implying a greater magnitude of multiplicity of illnesses in infants who had been assessed to have a relatively severe condition.

The utility of the ‘Referral Criteria’ outlined in ‘IMCI’ algorithm in predicting hospitalization was computed. The sensitivity of these criteria in total patients (n = 129) and follow-up certain subjects (n = 105) was 87% and 86% whereas the specificity was 58% and 53%, respectively. The positive predictive value, negative predictive value and odds ratio (95% CI) for the same were 75% and 78%, 76% and 66% and 9.3 (3.7-24.3) and 6.6 (2.4-18.9), respectively.

The morbidity profile observed as per the ‘gold standard’ is depicted in Table I. It is apparent that majority (81-84%) of the recorded diagnoses were either totally (40-44%) or partially (40-41%) covered by the ‘IMCI’ algorithms. Only a small proportion (16-19%) of the recorded morbidities was not covered by the algorithm.

Table I - ‘Gold Standard’ Morbidities Recorded

The diagnostic agreement was calculated between ‘gold standard’ and the ‘IMCI’ algorithm. A ‘total agreement’ was considered if the case required referral as per ‘IMCI’ algorithm and was actually admitted or if all the diagnoses made by algorithm matched the ‘gold standard’. Cases not fitting in the above cate-gory were defined as a mismatch. Table II summarizes the diagnostic agreement of the ‘IMCI’ with the ‘gold standard’. There was a total agreement on all diagnoses in a single patient in 60% to 66% of subjects. The mis-match was more commonly of an overdiagnosis (23% and 21%) rather than underdiagnosis (21% and 15%). If all referrals were considered a diagnostic match, the total agreement was 76% to 81%.

Table II - Summary of Diagnostic Agreement Between ‘Gold Standard and IMCI’

An attempt was made to identify presenting complaints of cases in which there was a diagnostic mismatch. The presenting complaints of these 52 and 36 cases are given in Table III. It is evident that cough, fever, coryza and diarrhea were the important presenting symp-toms for diagnostic mismatch. Upper respiratory infections were diagnosed in 63.4% and 52.7% of these patients and breastfed stools in 9.6% and 11.1%.

Table III - Presenting Complaints in Cases with Any Diagnostic Mismatch

+ Others included deviation of mouth, excessive crying, feeding difficulty, increasing head size, nasal block, rash, refusal of feeds, regurgitation of feeds and swelling back (all n = 1).

The sensitivity of algorithm to identify a serious (n = 57) or any (n = 64) bacterial infection was 96.5% and 96.8%, respectively while the specificity was relatively low (59.7% and 58.5%, respectively). The corresponding figures for the outcome certain subset (serious bacterial infection in 51 and any bacterial infection in 58) were 96.1% and 96.5% for sensitivity and 51.8% and 51.1% for specificity. Of the 43 cases identified as diarrhea, 6 (14%) had breastfed stools without diarrhea which did not need any therapeutic intervention. The algorithm performed well for detecting dehydration status (35/35) although it tended to overestimate the severity of dehydration in a few (6/35,17%) subjects.

With the IMCI algorithm, 55% of the non-referred subjects (17.8% of the total sample of 129 cases) would have received immunization. These subjects would have theoretically cons-tituted a missed opportunity for immunization if a solitary vertical program approach had been adopted. Also 55% of the non-referred cases (17.8% of the total sample of 129 cases) required counseling for appropriate breast-feeding practices. These children would also have represented a missed opportunity for counseling for appropriate breastfeeding with a solitary vertical program approach.

The current study reaffirms that co-existence of illnesses is a rule rather than exception even in young infants with three fourths of children having more than one illness. Similar observations in this age group were recorded in an earlier report(6). Another important finding was that the number of morbidities was higher in those children who had been assessed to have a relatively severe condition (means of 2.3 vs. 1.8 illness/child). Earlier data from Bangla-desh(6) in younger infants and from India nad Bangladesh(5,6) in older children also showed similar results.

An important component of ‘IMCI’ algorithm is the early recognition of severe morbidity requiring referral to a higher level of health facility for appropriate management. Problems in this area can easily undermine the confidence of the paramedical personnel and the community for this proposed health inter-vention. The IMCI guidelines are designed to be highly sensitive for the referral of patients with a possible severe illness, thus it inevitably leads to some children being referred un-necessarily(7). We compared the IMCI recom-mendation for referral with the judgement of senior pediatrician on the need for hospital-ization and found a reasonably good sensitivity (86-87%) but a lower specificity (53-58%). An earlier study from Bangladesh(6) in 234 young infants had also documented a higher sensitivity (84%) and a lower specificity (54%). Thus, there is a need to improve the specificity of the referral criteria. One possible area for improvement of excessive referral would be refinement to define upper respiratory infection since 13 of 21 unnecessary referrals (62%) in our study had this morbidity. The need for referrals for these 13 patients as per ‘IMCI’ algorithm was primarily fever. The Bangladesh study(6) also revealed that most frequent provisional diagnoses in patients unnecessarily referred were pneumonia as per WHO criteria (68%) and upper respiratory tract infections (13%). The addition of intercostal or supra-sternal retractions to lower chest wall indrawing increased the specificity of IMCI referral from 54% to 69% in young infants (27% increase, 95% CI: 8%-49%), while maintaining sensitivity.

Regarding overall performance of this algorithm, the clinical experience is limited in young infants. In the present study with ‘IMCI’ approach, there was a total agreement with ‘gold standard’ in all diagnoses and prescribed broad categories of treatments in a single patient in 59-66 per cent of subjects, if appropriate referral was considered a diagnostic match and in 76-81 per cent of subjects if all referrals were considered a diagnostic match. Thus, an import-ant reason for mismatch was excessive referral of the children who subsequently were not hospitalized. The algorithm does not have any provision for diagnosis of breastfed stools, which is a common occurrence in exclusively breastfed neonates. In the current study 14% of cases identified as diarrhea by IMCI (6/43 cases) had breastfed stools which would have resulted in unnecessary treatment for diarrhea. The IMCI algorithm also focuses on the provision of preventive services like immunization and feeding advice for every child, which tend to get ignored with disease specific vertical algo-rithms. In the current study, there was a possi-bility of missed opportunities for immunization and breastfeeding counseling, which were effectively covered by the ‘IMCI’ in 18% of subjects for both.

The algorithm had a high sensitivity (96.1-96.5%) but a lower specificity (51.8-59.7%) for identifying a serious bacterial infection. A recent multicentric trial(8) has attempted to generate more accurate clinical predictors of serious bacterial illness in young infants. This study compared the accuracy of the generated 14 item simplified three-level model with that of the 12 clinical signs in the WHO guidelines for the management of the sick young infants. In detecting infants under 60 days of age with any outcome abnormality, the WHO sick child criteria had an ROC area of 0.656 as compared to 0.838 for the three-level model, signifying greater accuracy of the latter. This research may permit the development of more accurate guidelines for management of sick young infants.

It would be prudent to recall that the present study quantified the utility of the ‘IMCI’ algorithm on the basis of an assessment under-taken by a resident undergoing pediatric postgraduate training and the efficacy is likely to diminish in the hands of trained para-medical personnel. Also, the study was conducted in an urban tertiary care center and may not be representative of the whole community. In this context, it would be desirable to evaluate the actual performance of trained para-medical personnel in the true field setting. The ‘IMCI’ algorithm is complex and there is a concern about the difficulty health workers had in using it, and the potential for longer consultation times(9,10).

In conclusion, there is a sound scientific basis for adopting the ‘IMCI’ approach even in the age group one week to two months since co-existence or morbidities is a rule rather than exception and the algorithm provides good sensitivity for assessing severe illness. However, the specificity for prediction of severe illness requires improvement. Two important conditions identified for possible refinement are upper respiratory tract infection and breastfed stools.

Contributors: HPSS coordinated the study (particularly its design and interpretation) and drafted the paper, he will act as guarantor for the paper. RG participated in the data collection, and also helped in drafting the paper. DS participated in analysis and drafting.

Funding: None.
Competing interest: None stated.

1. WHO. Perinatal mortality: A listing of available information. Maternal Health and Safe Motherhood Programme, Geneva, World Health Organization, 1997.

2. Ramji S, Sachdev HPS. Fertility and mortality indicators. Indian Pediatr 1996; 33: 877-881.

3. World Health Organization. Integrated management of the sick child. Bull WHO 1995; 73: 735-740.

4. Gove S. Integrated management of childhood illness by outpatient health workers: Technical basis and overview. Bull WHO 1997; 75 (Suppl 1): 7-24.

5. Shah D, Sachdev HPS. Evaluation of the WHO/UNICEF algorithm for integrated management of childhood illness between the age of two months to five years. Indian Pediatr 1999; 36: 767-777.

6. Kalter HD, Schillinger JA, Hossain M, Burnham G, Saha S, deWit U, et al. Identifying sick children requiring referral to hospital in Bangladesh. Bull WHO 1997; 75 (Suppl 1): 65-75.

7. Gove S. Integrated management of childhood illness: Conclusions. Bull WHO 1997; 75 (Suppl 1): 119-128.

8. The WHO Young Infants Study Group. Clinical predictors of serious bacterial infection in young infants in developing countries. Pediatr Inf Dis J 1999; 18 (Suppl 10): S4-S7.

9. Simoes EAF, Desta T, Tessema T, Gerbresellasie T, Dagnew M, Gove S. Performance of health workers after training in integrated management of childhood illness in Gondar, Ethiopia. Bull WHO 1997; 75 (Suppl 1): 43-53.

10. Kolstad PR, Burnham G, Kalter HD, Kenya-Mugisha N, Black RE. Potential implications of the integrated management of childhood illness (IMCI) for hospital referral and pharmaceutical usage in Western Uganda. Trop Med Int Health 1998; 3: 691-699.