Home            Past Issues            About IP            About IAP           Author Information            Subscription            Advertisement              Search  

   
review on child health priorities

Indian Pediatr 2011;48: 183-189

UNICEF-PHFI Series on Newborn and Child Health, India: Methodology for Systematic Reviews on Child Health Priorities for Advocacy and Action


Joseph L Mathew, *Dheeraj Shah, Tarun Gera, $Siddhartha Gogia, **Pavitra Mohan,
$$Rajmohan Panda, $$Subhadra Menon, and *Piyush Gupta

From Advanced Pediatrics Center, PGIMER, Chandigarh; *University College of Medical Sciences, New Delhi;Fortis Hospital, New Delhi; $Max Hospital, Gurgaon, Haryana; **UNICEF, India; and $$Public Health Foundation of India, New Delhi, India.

Correspondence to: Joseph L Mathew, Advanced Pediatrics Center, PGIMER, Chandigarh 160 012, India.
Email: [email protected]

 


India is committed to reducing childhood mortality and morbidity. This requires evidence-based policy and practice in the realm of public health. This in turn necessitates advocacy and action (among all stakeholders), focused on locally relevant issues. A collaboration to work towards this goal was forged between the Public Health Foundation of India (PHFI), United Nations International Children’s Emergency Fund (UNICEF), India; and a team of independent researchers. As a first step, a systematic review of literature on four priority areas of newborn care (community-based interventions) and child health (acute respiratory infection, diarrheal disease, anemia), was undertaken to address important issues including epidemiology, interventions for management, and operational issues of planning, implementing, and measuring actions at a programmatic level. This paper describes the development of the methodology for undertaking these systematic reviews including the process for framing of research questions, building a research team, and executing the systematic review (literature search strategy, data extraction, analysis, and reporting). The challenges associated with ensuring robust methodology, are also described.

Keywords: Action, Advocacy, Anemia, ARI, Child health, Community based newborn care, Diarrhea, India, Methods, Newborn, Systematic reviews.


Reducing childhood mortality and forging improvement in Newborn and Child Health (NCH) are among India’s most important public health goals [1]. The importance of generation of evidence to inform policies and programs for achieving sustainable gains in line with our country’s goals as well as the global Millennium Development Goals [2] cannot be over-emphasized. In the era of evidence-based health-care and informed health-care policy-making, all initiatives should be developed on a solid foundation of evidence. The Public Health Foundation of India (PHFI) collaborated with UNICEF to generate an agenda for advocacy and action on key technical and programmatic issues related to newborn and child health survival in India. It was proposed to generate the evidence through systematic reviews, discussions with experts and consultation with stakeholders. Based on initial discussions in January 2010, four key areas were prioritized for the review: community based newborn care, childhood anemia, acute respiratory infections (ARI) and diarrheal diseases. This paper describes the methodology employed for the identification, synthesis and collation of evidence in these four key areas of newborn and child health in India.

The nature of the questions addressed (which included technical as well as operational issues) and the intended purpose (to pursue advocacy and action at a programmatic level) necessitated the development of a separate methodology for this set of systematic reviews. Standard systematic review methodology (based on conventional literature searching) is designed to answer clinical questions (particularly interventions) in health-care, hence was not expected to be sufficient for this review.

The Review Process

Formulating Review Questions

A two-day consultative meeting of research experts was organized on 9-10 February 2010 at New Delhi, under the aegis of UNICEF, New Delhi, India and PHFI, to formulate questions for review from each of the 4 thematic areas. Five to six experts per thematic group were invited to formulate and refine the review questions. The approach was aimed to systematically and comprehensively list the relevant questions, and give equal treatment to questions in different domains: epidemiology, preventive and therapeutic interventions, health systems and health policy. Responses from these experts were discussed in a Delphi like process to refine, sequence and prioritize the review questions [3,4]. The final list of questions for each thematic area was generated summarizing the input from the experts, following the discussions for refining each research question, under a moderator. Within each question, one or more sub-questions was/were also included. We grouped the final list of questions under two domains: Technical issues; and Operational issues. Since many of the operational issues were cross-cutting, these were further refined to maintain uniformity across the thematic areas. The final list of questions generated is presented in Tables I-IV.

Table I

Acute Respiratory Infections (ARI): Questions Reviewed

Technical issues Operational issues

• What is the current status (and trends) of morbidity and mortality from childhood ARI. Identify risk/predisposing factors contributing to burden of ARI/mortality due to pneumonia. What proportion of ARI cases have pneumonia/severe pneumonia?

• What is the common etiology of community acquired ARI, pneumonia and severe pneumonia? (India specific data and trends)? What is the anti-microbial susceptibility pattern and mechanism of surveillance for antibiotic resistance?

• What are the current National guidelines for management of ARI (both at community and facility levels)? What is the role of potential interventions (case finding and community based management, antibiotics, zinc, vitamin A, measles, pertussis, Hib and pneumococcal vaccine, supportive management and oxygen, environment) that may result in reduction of mortality/prevention of pneumonia?

• What proportion of ARI cases have wheeze (audible/auscultation)? Is it predictive of severe pneumonia/mortality/hypoxia? What proportion of cases with wheeze respond to bronchodilator therapy? What is the impact of treating a wheeze related ARI with/without bronchodilators?

• What are the client practices regarding management of ARI (care seeking, home remedies)?

• What are the current prescription practices for antibiotics, cough remedies, bronchodilators etc. in management of ARI?

• What are the existing strategies/policies/programs at national/State levels? What are the experiences? What are the indicators for monitoring/evaluation? And what are the barriers (both from and within ‘System’ and ‘Demand’ side, and also technical/managerial) to implementation of standard ARI case management? (these will include procurement, distribution, advocacy, managerial spects, KAP, compliance etc).

• What are the experiences with integrated vs vertical approach (in program mode)?

• What is the role of community health worker in ARI/pneumonia control program? Are they equipped and empowered to manage ARI? How do they perform after training? (Global experiences)

• What is the feasibility of diagnosis and management of wheeze in the community?

 

Table II

Diarrheal Disease: Questions Reviewed

Technical issues Operational issues

• What is the magnitude/trend of diarrheal morbidity (incidence) and mortality in India?

• What is the etiology (and trends therein) of diarrhea, including neonates and young infants?

• What are the current National guidelines for management of diarrhea including the role of low osmolarity ORS and Zinc in management of diarrhea?

• What is the role of other interventions (breastfeeding, immunization, vitamin A/zinc supplementation, hand washing, environmental modification) in prevention of diarrhea?

• What are the client practices regarding management of diarrhea (ORT, feeding, care seeking)?

• What are the current prescription practices for ORS/ORT, Zinc, antibiotics, pre-probiotics, antisecretory agents, antiemetics, other antidiarrheal drugs (metronidazole etc) in management of diarrhea?

 

• What are the existing strategies/policies/programs at national/State levels? What are the past and present experiences? What are the indicators for monitoring/evaluation? And What are the barriers (both from and within ‘System’ and ‘Demand’ side, and also technical/managerial) to implementation of the program? (these will include issues related to procurement, distribution, advocacy, managerial aspects, KAP, compliance etc.)

• What are the experiences with integrated vs. vertical approach (in program mode)?

• What are the use rates of ORS and Zinc and barriers for their use?

Formulation, branding, prescription, dispensing, quality, manufacturing capacity, social marketing, and availability of ORS/zinc

IEC, advocacy material, capacity building

Private sector/ AYUSH involvement

Current evidence/experience related to compliance of zinc use for 10-14 days.

• How to promote large scale use of ORS and Zinc rapidly? (Global Experiences)

Table III

Childhood Anemia: Questions Reviewed

Technical issues Operational issues

• What is the prevalence of childhood anemia in India and which are the groups at risk? Identify the target groups for prevention and control of anemia.

• What are the major etiologic factors contributing to anemia in children in reference to various regions of India? (a) Nutritional iron, B12, FA, dietary habits, bioavailability, PEM, others; (b) Maternal anemia (life cycle approach); (c) Helminth infestation, H. pylori; (d) Malaria; and (e) Hemoglobinopathies.

• What are the functional and economic consequences of anemia in children, specially on (a) Mental and motor development; (b) Physical growth; (c) Physical Capacity; (d) Under-5 mortality; and (e) Infection?

• What are the available strategies/models for anemia control and prevention, including (a) Maternal anemia control programs (Life cycle approach); (b) Exclusive breastfeeding; (c) Deworming; (d)Supplementation programs (Dose, Duration, Formulations, Daily vs. intermittent, Iron vs. multimicronutrient); (e) Food based approach; and (f) Fortification? What is the effectiveness of different strategies to prevent and control anemia among children, particularly with reference to impact on functional consequences?

• What are the adverse effects and safety aspects of supplementation programs?

• What are the existing strategies/policies/programs at national/State levels? What are the experiences? Is National Anemia control program sufficient in concept and implementation? Was it adequately managed? Whether the program has been sufficiently evaluated and improvised/modified based on the evaluation feedback? Has the cost effectiveness ascertained? What are the indicators for monitoring/evaluation? And What are the barriers (both from and within ‘System’ and ‘Demand’ side) to implementation? (these will include procurement, distribution, advocacy, managerial aspects, KAP, compliance etc.)

• What are the experiences with integrated vs. vertical approach (in program mode)?

• What can be done to scale up the effective anemia prevention and control strategies?

 

Table IV

Community-based Newborn Care: Questions Reviewed

   

• What is the magnitude of neonatal deaths in India and what is the etiology. Are there any changing trends?

• What are the packages of interventions (Typology? Timing?) that have been used for community newborn care?

• What has the effect of interventions been in terms of outcome – mortality, neonatal care practices and health seeking behavior?

• Are the studies comparable in terms of interventions, baseline neonatal mortality, training of health workers, type of workers, supervisory support, facility and referral support and the population size of the area where intervention took place?

• What are essential elements of home and community based care that makes the most difference to neonatal health outcomes?

• What are the existing strategies/policies/programs at national/State levels? What are the experiences? What are the indicators for monitoring/evaluation? And what are the barriers (both from and within ‘System’ and ‘Demand’ side) to implementation? (these will include procurement, distribution, advocacy, managerial aspects, KAP, compliance etc.)

• What is the profile of the service provider in different models? What training methods have been used for health workers/supervisors for community newborn care interventions? Which skills have poor retention amongst workers?

• Under what circumstances are these interventions likely to have an impact? (Supervision, Logistics and Support systems, Motivation of health providers)

• Have any of these interventions in studies been taken to scale and do we have a measure of the effectiveness? What are the difficulties in taking these interventions to scale? Which of the interventions within the package have been difficult to implement? Cost effectiveness?

Research Team

To identify current evidence on the subject, a research team comprising of one Coordinator, four Authors and four Reviewers (one per topic area) was constituted. The team also included experts from UNICEF, New Delhi and the Public Health Foundation of India.

Formulating Methodology

Following identification of questions, the Authors developed a research protocol building up on the initial suggestions provided by the experts. The initial protocol was prepared on 23 Feb 2010; it was modified, refined, revised and finalized on 31 March 2010. Pilot testing of this version was undertaken by each Author and the results presented in a Consultative Meeting held on 6 April 2010. At this meeting, the methodology for the systematic reviews was presented to all the members of the research team including UNICEF and PHFI partners. Based on feedback received, minor modifications were made which was then finalized. The Methodology is briefly described below (also see Web Table). Each systematic review was undertaken by the respective Author based on the Methodology agreed upon.

Sources of Literature

To address the technical issues (epidemiology, risk factors, efficacy of interventions, client and professional practices, etc), the primary databases employed were Medline through Pubmed and the Cochrane Library (http://www.thecochranelibrary. com/view/0/index.html). It has been shown that a brief RCT search strategy comprising these databases is generally sufficient to locate trials for systematic reviews in most cases and exhaustive searching is no longer regarded cost-effective, if the most important databases have been covered [5]. However, it has also been recommended that search to other sources, particularly regional databases is a preferred option [6]. Therefore, IndMed (http://indmed.nic.in/) was also included as a primary database. The Authors also accessed specific sources to address specific questions within each thematic area. Some of these sources contained generic information cutting across the four thematic areas; including the National Family Health Survey (NFHS) website for the three NFHS reports (www.nfhsindia.org), Sample Registration System (censusindia.gov.in/Vital.../SRS/Sample_ Registration_System.aspx) for data on (cause-specific) mortality, etc. Some of the sources/databases were specific for a thematic area, for example the National Nutrition Monitoring Bureau (www.nnmbindia.org) for childhood anemia. The Authors were free to search any other additional database with a specific justification for the same.

For the operational issues, it was recognized from a pilot test-search that the literature sources listed above would be inadequate; hence, the following additional sources were accessed: Popline.org (www.popline.org), documents published by the World Health Organization and available online (www.who.int), documents of the UNICEF available online (www.unicef.org/india/), documents of the Government of India available online, specific publications of the Ministry of Health and Family Welfare (www.mohfw.nic.in), publications of the Indian Council of Medical Research (www.icmr.nic. in), and Central Bureau of Health Intelligence (www.cbhidghs.nic.in/) reports. In addition, the Authors were encouraged to search other literature sources including textbooks, abstract books, conference proceedings, etc with justification for the same. Authors were free to contact related subject experts. At the outset, it was realized that the broad range of issues to be addressed and the type of questions raised would necessitate a systematic review process, far beyond the usual reviews undertaken to evaluate efficacy of interventions [7]. The usual PICO (population, intervention, com-parator, outcome) framework [8,9] would not be the best solution for many of the questions, especially the Operational issues. In order to capture the vast amount of data likely to be available, and at the same time retain (methodological) quality, it was decided a priori to use the process described below.

Inclusion and Exclusion criteria

For the technical issues, the emphasis of literature search was to collect and collate current, best evidence on the specific question under review. The Technical questions themselves were broadly categorized as Epidemiology questions (magnitude, time-trend, geographical trends, social/cultural trend) and Intervention questions (prevention or treatment). For the former, the following hierarchy was used: I: Systematic review of interventional trials published within the last 10 years (in the Cochrane reviews, the date until which literature had been searched is also available); II: Randomized controlled trial(s) (if systematic review(s) within the past ten years, was/were unavailable); III: Quasi-randomized trial (if no randomized trial(s) was/were available); IV: Non-randomized trial (if neither a systematic review nor randomized trial nor quasi-randomized trial were available); V: Modelling/ secondary calculations based on studies within the past ten years (if the preceding study types were not available); VI: Modelling/ secondary calculations based on studies older than ten years (if none of the above was available).

For the Intervention questions, the following hierarchy was used: I: Systematic review of interventional trials within the last 10 years; II: Randomized controlled trial(s) (if no SR available within the past ten years); III: Quasi-randomized trial (if no randomized trial was available); IV: Non-randomized trial (if no SR or RCT was available); V: Modelling/ secondary calculations based on studies within the past ten years (if the above study types are not available); VI: Modelling/ secondary calculations based on studies older than ten years (if none of the above was available). Literature search using this hierarchy ensured that high quality evidence was preferentially sought and included; at the same time, it allowed different study designs to be included; without resorting to data likely to be biased [5]. Table V summarizes the Methodology process.

A similar hierarchy could not be utilized for literature search on operational issues. Hence it was decided a priori that no specific order of preference would be applied. Nevertheless, the following were preferentially sought and included: (i) documents originating from/ pertaining to India, (ii) documents originating from developing countries, (iii) documents pertaining to developing countries in general, (iv) documents originating from/pertaining to developed countries or unspecified countries, that could have a bearing on India and/or developing countries, based on the content presented. It was also decided a priori that among the documents retrieved, those which presented a Methodology section would be given higher preference, as the risk of bias is lower; and among those where Methodology was presented, those documents with robust Methodology were accorded greatest preference. For both sets of questions, only English language publications were sought and included.

Search Strategy and Selection of Publications

For the technical issues, each question necessitated a separate search string. The question was converted to the PICO format and key words selected for searching. The specific search terms used are presented in the four individual reviews. Each search was designed as a "broad sensitive" search, and if required tapered to a "narrow specific" search. For the Operational issues, various search terms were tried singly and in combination, to identify the string with the maximum output. After several trials, the following string was selected and set for all the 4 reviews: * AND india AND (health policy OR health planning OR health programs OR health services OR program evaluation OR operations research) where the asterisk represents the topic under review (ARI, diarrhea, anemia, or community-based newborn care).

The search date, search terms, search string and search output were recorded and saved. The following format was used by each Author to consider publications for inclusion/exclusion in the review: (i) Examination of title; those titles obviously not relevant were excluded and the rest processed further; (ii) Examination of Abstract or Introduction (where abstract was not published) of the short-listed titles; those which were not relevant were excluded; (iii) Examination of full-text; those publications which did not match the inclusion criteria were excluded; and the remainder processed to the next filtration step; (iv) Amongst the selected publications, the following were included: those pertaining to the pediatric age group (for questions relating to epidemiology, interventions and programmes), for interventions, standard dose/route/administration modalities were preferentially included over experimental and/or special models in research settings; and those with primary/hard outcomes were preferentially included over secondary/surrogate/soft outcomes for interventions (especially treatment and/or prevention modalities).

Data Collection and Analysis

The principal author for each topic studied each included publication in detail and extracted data relevant to the review. In case of anything lacking clarity, the Reviewer for the topic examined the publication. If further clarification was required, the team of four principal investigators deliberated on the issue and arrived at a consensus.

Extracted data were synthesized in a descriptive manner. It was decided a priori that no secondary data analysis (meta-analysis or other statistical tests) would be performed, since the objective of the systematic review was to identify issues for advocacy/action; and the types of publications retrieved in each thematic area were likely to preclude pooled analysis of aggregate data through meta-analysis [10]. However, for the review on community-based newborn care, a sub-group analysis was done.

There was considerable discussion on whether or not to critically appraise each included publication for methodological quality. Based on the facts that multiple study types would be eligible for inclusion; high degree of variability in components, construction and properties of critical appraisal tools for research reports and absence of a "gold standard’ critical appraisal tool for any study design [11], it was decided that no additional critical appraisal (except for Methodology i.e validity) would be undertaken. Where possible, numerical data were extracted and presented in a tabular format. Where essential, a footnote for clarification was included.

Although there is a risk of publication bias when only published literature is included; many systematic reviews themselves largely do not recognize the impact of the problem [12]. Therefore, it was decided that no statistical tests for publication bias would be undertaken; and no additional searches for unpublished literature would be undertaken to counter the potential of the same.

Quality Assurance

On completion of the four systematic reviews, each underwent a first-level review by Reviewer with experience and expertise in the topic under review as well as expertise in research methodology. Based on feedback received, the reviews were modified, revised and finalized. The detailed findings were presented at a third Consultative Meeting held on 4-5 May 2010 at Manesar, Haryana. This was attended by the Research Team, Senior Reviewers and partners from UNICEF and PHFI. Based on feedback from all participants, the four reviews underwent further revision and modification. A fourth Consultative Meeting was held on 21 July 2010 at New Delhi, which was attended by (amongst others) Dr H Hombergh, Chief, Health, UNICEF India Country Office and Prof K Srinath Reddy, President PHFI. The key findings of the four systematic reviews were presented, additional feedback received and the reviews were finalized.

The finalized systematic reviews were prepared for publication in accordance with the standard guidelines for publication of research papers. The first draft was prepared with a deadline of 15 August 2010, revisions were made and finalized by 15 October 2010.

Contributors: JLM devised the methodology, drafted and finalized the manuscript. DS and PG provided important additional inputs to Methodology and revised the manuscript. All authors contributed to the process of finalizing the Methodology and manuscript.

Funding: UNICEF.

Competing interests: None stated.

Disclaimer: Opinions expressed by PM, RP and SM are their individual views and do not in any way reflect upon the institutions, they serve.

References

1. Ministry of Health and Family Welfare (Government of India). Child Health Programme in India. Available from: http://www.mohfw.nic.in/dofw%20website/child%20healthrti.pdf. Accessed November 11, 2010.

2. United Nations Millenium Development Goals. Available from: http://www.un.org/millenniumgoals. Accessed November 11, 2010.

3. Byrne S, Wake M, Blumberg D, Dibley M. Identifying priority areas for longitudinal research in childhood obesity: Delphi technique survey. Int J Pediatr Obes. 2008;3:120-212.

4. Adler M, Ziglio E (eds). Gazing into the Oracle: The Delphi Method and its Application to Social Policy and Public Health. London: Jessica Kingsley Publishers; 1996.

5. Smith LA, Moore RA, McQuay HJ, Gavaghan D. Using evidence from different sources: an example using paracetamol 1000 mg plus codeine 60 mg. Available from: http://www.biomedcentral.com/1471-2288/1/1. Accessed November 10, 2010.

6. Betrán AP, Say L, Gülmezoglu A, Allen T, Hampson L. Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality. Available from: http://www.biomedcentral.com/1471-2288/5/6. Accessed November 10, 2010.

7. Straus S, Haynes B, Glasziou P, Dickersin K, Guyatt G. Misunderstandings, misperceptions, and mistakes. Evid Based Med. 2007;12:2-3.

8. Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90:837-40.

9. Mathew JL, Singh M. Evidence based child health: fly but with feet on the ground! Indian Pediatr. 2008;45:95-8.

10. Lyman GH, Kuderer NM. The strengths and limitations of meta-analyses based on aggregate data. Available from: http://www.biomedcentral.com/1471-2288/5/14. Accessed November 10, 2010.

11. Katrak P, Bialocerkowski AE, Massy-Westropp N, SaravanaKumar VS, Grimmer KA. A systematic review of the content of critical appraisal tools. Available from: http://www.biomedcentral.com/1471-2288/4/22. Accessed November 8, 2010.

12. Phillips CV. Publication bias in situ. Available from: http://www.biomedcentral.com/1471-2288/4/20. Accessed November 13, 2010.

 

Copyright© 1999 by the Indian Pediatrics (Disclaimer)