An external evaluation of the Program was conducted to obtain an independent, objective and credible assessment of the strengths and weaknesses of the program that would help INCLEN and BMGF in making amendments to their programmatic operations, and to also enable other stakeholder and research funders to take informed decisions in supporting future research projects in India. This report briefly summarizes the methods, key findings and the recommendations from the evaluation.

The authors of this paper were part of the three-member external evaluation team, which conducted the evaluation from 1-10 April, 2019. The evaluation focused on addressing the following four questions related to the overall program objectives: i) Was the research conducted under the Program of public health relevance? ii) Were the outputs from the Program likely to influence public health policy and programme planning and implementation? iii) Did the Program result in broadening of the researcher pool? iv) Did the Program strengthen research capacity in India?

The evaluation consisted of a desk review of all available documents and reports provided by the INCLEN secretariat, visits to seven of the ten research sites, telephone or Skype interviews with investigators at the three remaining research sites (Fig. 1), and interviews with a sample of TAG members, members of the INCLEN secretariat, and BMGF.

Fig. 1 Site visits and interviews with research investigators.

Based on the geographic spread and the range of investigators who responded to the Request for applications (RFA), it appears that this process was successful and achieved its objective of securing applications from around the country.

The evaluation indicated that the processes for the selection of research projects was transparent and objective for the first round of application. The choice of a single study in the second round was influenced by the funder’s priorities. The process managed to draw in less-experienced investigators and strengthened their capacity to conduct high-quality research. The provision of financial support to institutions that did not have clearance from the government under the Foreign Contribution Regulation Action (FCRA) to accept international funds was a significant achievement of this program, which enabled researchers from such institutions to carry out more substantive research projects, which they may not otherwise have been able to conduct.

All the selected proposals were of public health relevance in India. The close engagement and support provided through the JWG and TAG resulted in improved study design and implementation of most of the studies. The program also created a new paradigm for the conduct, oversight and engagement with individual research studies in India to optimize their outputs and impact. The support from the INCLEN secretariat, both technical and administrative, was greatly appreciated by all the investigators. The INCLEN secretariat also conducted a Research leadership and programme management (LAMP) workshop, which benefited many of the new researchers in the program.

The program did have a few weaknesses, primarily that the expertise of the JWG and TAG was not fully exploited. Closer engagement of these two bodies during the implementation of the Program may have resulted in better outcomes for a few of the studies. While the TAG was closely involved in the process of selection of projects and in the finalization of study design, its engagement thereafter in the monitoring process was patchy. In addition, protocol deviations or changes were not always conveyed to or endorsed by the TAG as envisaged. The monitoring by the INCLEN secretariat largely served to keep the projects on track and facilitated support for overcoming procedural bottlenecks. However, this oversight missed a few technical issues and protocol deviations that impacted the study outputs in a few projects.

Capacity strengthening was an important contri-bution made by the program. From all reports, the process and support provided far exceeded that provided by other national and international grant-making bodies and served in improving the design and implementation of the studies and resulted in better outputs. This was especially true for the less experienced investigators, though even the more experienced investigators reported having benefited from their participation in the program.

Of the ten research studies, seven are likely to make significant contribution to public health policy and program implementation in India. However, INCLEN may need to play a greater role in disseminating the results to policymakers and program managers to promote the translation of the study results into public health policy and practice. Four studies evaluated care-seeking behavior for pneumonia, of which two have been published [2,3]. Two of them, which were mixed-method studies, found that knowledge about pneumonia was poor and care was mostly obtained from private service providers who were not knowledgeable about pneumonia case management. Two other studies evaluated the impact of behaviour change communications (BCC), one which looked at its impact on the incidence of childhood diseases including pneumonia and the other that combined BCC with capacity strengthening for case management in primary care facilities in the public sector. Taken together, the findings of these studies can make important contributions to improving care-seeking and the quality of care for childhood pneumonia if it is possible to translate the findings into public health practice.

A fifth study evaluated the use of high flow nasal cannula delivery of oxygen for children with pneumonia and found that the technique and could be implemented in first referral level facilities could reduce the need for invasive ventilation, which is only available in tertiary care facilities. The sixth study used mathematical modelling to estimate the state level burden of pneumococcal disease and the cost-effectiveness of pneumococcal conjugate vaccine (PCV). The preliminary findings from this study informed the policy recommendations for the use of PCV in India. A seventh study, which is yet to be completed, is also likely to inform both global and national policies on the use of PCV.

The evaluation team made recommendation related to the current projects as well as for future projects supported by INCLEN; these are summarized below.

Contributors: TC, PG, KT: evaluation undertaken and findings were analyzed and interpreted; TC: initial draft report was penned; KT,PG: intellectual inputs. All authors have approved the final report and take full responsibility for its contents.

Funding: This work was supported by Bill and Melinda Gates Foundation through The INCLEN Trust International (Grant number: OPP1084307). The funding source had no contribution in study design, implementation, collection and interpretation of data and report writing. Competing interest: None stated.

1. Research program to emphasize public health significance of childhood pneumonia in India. 2017 [cited 2019]. Available from: http://inclentrust.org/inclen/research-program-to-empha size-public-health-significance-of-childhood-pneumonia-in-india/. Accessed 17 December, 2019.

2. Awasthi S, Verma T, Agarwal M, Pandey CM. To assess the effectiveness of various communication strategies for impro-ving childhood pneumonia case management: Study protocol of a community based behavioral open labeled trial in rural Lucknow, Uttar Pradesh, India.　BMC Pediatr. 2018;18:279.

3. Mohanraj R, Kumar S, Jayakumar S, Agarwal M, Dhingra B, Jeyaseelan V, et al. Where do mothers take their children for pneumonia care? Findings from three Indian states.　PLoS One. 2019;14:e0214331.