Despite rising rates of institutional delivery, most neonatal deaths in India occur at home. This also holds true in the state of Uttar Pradesh where 27% of neonatal deaths occur. The Government of India (GOI) released home-based newborn care (HBNC) guidelines in 2011 to increase access to newborn care through accredited social health activists (ASHAs) [1]. The guidelines expect ASHAs to make home visits to promote essential newborn care, identify illness, and refer infants if needed. ASHAs receive a performance payment for conducting the visits [2].

Government of Uttar Pradesh (GoUP) HBNC guidelines require ASHAs to use three formats to record information and assess and classify illness according to decision support algorithms: Format-1 is for the mother; format-2 for newborn examination and format-3 Integrated Management of Neonatal and Childhood Illness (IMNCI)-based color codes to diagnose and classify sick infants [3]. The ‘Manthan’ Project, led by IntraHealth International and funded by the Bill & Melinda Gates Foundation, completed a situational analysis in Jhansi District to evaluate the performance of IMNCI-trained ASHAs to use HBNC guidelines to detect and assess illness severity in infants.

This four-month study (November 2012-February 2013) was conducted in the service areas of two additional primary health centers (APHCs) in Babina block. Twenty-five IMNCI-trained ASHAs participated. The ASHAs received a 5-day refresher training that included orientation on GoUP HBNC guidelines, instruction on data recording formats, and an IMNCI skills review. Three trained postgraduate female investigators observed ASHAs during home visits. The GoUP and IntraHealth’s Institutional Review Board approved the study. ASHAs provided written consent, and investigators obtained informed verbal consent from mothers of assessed newborns.

Investigators observed 47 home visits (roughly two per ASHA). Two visits were excluded from the analyses because they were incomplete, leaving 45 eligible visits. ASHAs visited newborns under two months (1-42 days). Investigators used a checklist to record their skills, and also completed their own assessments. Stata 10.0 was used to measure sensitivity and specificity for each sign and symptom for identifying illness. Kappa statistics calculated ASHA-investigator agreement. Categories for the agreement unweighted kappa statistics were: poor (< 0.40), intermediate (0.40 - 0.75), good (> 0.75 - 0.90), and excellent (> 0.90).

Overall, ASHAs did not complete comprehensive assessments using format-2. Temperature was assessed in 83% of observations, and correct steps followed in less than half. Weight was assessed in 87% of observations, and all correct steps followed in 30%. Although ASHAs inquired about breastfeeding in nearly three-fourths of visits, they assessed critical breastfeeding issues (e.g., difficulty feeding, decreased milk supply) in only one-third of observations. ASHAs examined the cord in 71% of cases but were less likely to perform other physical examinations such as examining chest indrawing (42%) or skin cracks or redness (38%). Assessment for most danger signs was low; ASHAs examined infants for lethargy or unconsciousness in 33% of cases and assessed jaundice or abdominal distension in 24% and 18% of cases, respectively.

ASHA-investigator agreement on the need for further assessment of infants (format-3) was intermediate (kappa 0.48, P=<0.001). Investigators identified the need for further assessment in 14 cases whereas ASHAs identified nine (78% agreement). Signs with complete ASHA-investigator agreement included jaundice, chest in- drawing, and lethargy or unconsciousness (100% specificity). Disagreement occurred for skin pustules, cracked nipples or engorged breasts, and reduced breastfeeding. ASHAs recognized two breastfeeding difficulty cases compared with 11 identified by investigators. There was considerable disagreement for incessant crying or infrequent newborn urination (80% sensitivity).

TABLE  I	Asha-investigator Agreement in Integrated Management of Newborn  And 

Childhood Illnesses (Imnci) Categorization of Infants 

Table I shows infants’ distribution in IMNCI illness categories, comparing ASHA versus investigator classifications. Overall agreement was poor (kappa=0.23, P=0.01). ASHAs misclassified 4 out of 5 infants belonging in red as yellow or green (80%), and placed 7 out of 8 infants into green though they belonged in yellow (88%). ASHAs classified 40 cases as green (versus 32 by investigators), indicating some underdiagnosis of illnesses.

None of the 25 trained ASHAs comprehensively covered all questions and signs, and they often skipped assessment items. ASHAs were more likely to ask about breastfeeding, newborn warmth, and crying but less likely to examine or assess for danger signs. ASHAs had difficulty sequentially asking questions and examining newborns while simultaneously recording information. This left critical assessment gaps and increased underdiagnosis of comorbidities, as reflected in the intermediate ASHA-investigator agreement on the need for further infant assessment.

The poor agreement between ASHA and investigator assessments of illness severity (kappa=0.23, P=0.01) has serious implications for efforts to reduce neonatal mortality, given that delays in referring sick newborns can prove fatal. Similarly, misclassification into green of infants who belong in yellow could result in withheld treatment and rapid progression of illness. Some ambiguity in format-2 instructions and duplication in format-2 and format-3 items may have created confusion about the assessment process. This suggests the need for revising the formats to eliminate repetition and make them user-friendly.

Acknowledgments: National Rural Health Mission, Government of Uttar Pradesh, for the support and cooperation extended to make this study possible and the Bill & Melinda Gates Foundation for financial and technical support. Dr France Donnay (Bill & Melinda Gates Foundation) and Dr ME Khan (Population Council) for their inputs, critical review and valuable insights through the study.　Manthan Project staff and colleagues at IntraHealth International.

Contributors: All authors participated in conceptualizing the study. ED: developed the analytical plan, analyzed the data, searched literature, and drafted and revised the manuscript. She will act as guarantor of the study. DSP: interpreted data and drafted, edited, and revised the manuscript; EAF: edited and revised the manuscript; GB: interpreted data and revised the manuscript; and MCC: searched literature and edited and revised the manuscript. All authors approved the final manuscript.

Funding: The Bill & Melinda Gates Foundation (BMGF), Competing interests: None stated.