The USAID Applying Science to Strengthen and Improve Systems (ASSIST) Project supported health workers in over 400 public health facilities in six states in India to use quality improvement (QI) approaches to deliver better care [5]. Health workers in these facilities were trained in the use of QI methods during a one-day classroom training followed by monthly on-site coaching visits to help them apply what they had learned to solve problems in quality of care to their settings. Teams were initially asked to work on improving routine elements of care that were required by all women or newborns. Once health workers had learned how to use QI methods for these types of problems, they were supported to move onto more complex elements of care, including handling complications.

We aimed to analyze the association between the presence of a QI team working on improving a specific element of routine delivery, post-partum or post-natal care and the probability that women and babies received that element of care. We also assessed the association between different health system factors and the probability of women or babies receiving appropriate care.

The current study to identify factors associated with clinical quality was built into a study to determine the validity of data on clinical care recorded by health care providers. The study was conducted in 12 public health facilities across six states (two facilities per state; one district hospital and one community health center), being supported by the USAID ASSIST project to use QI approaches to deliver better care around the time of delivery. The selection of facilities was purposive based on ease of access and having received one day of QI training and at least six QI coaching visits (i.e., the facility staff had been trained in QI at least six months ago). To identify a 5% difference between the recorded and observed data, we planned to observe 444 deliveries (74 per state) [6]. We planned to observe 55 deliveries at one district hospital (DH) and 19 at one community health center (CHC) in each state (3:1 ratio) with half observed during the day shift and half during the night shift. The elements of care that we assessed were: (i) administration of oxytocin within one minute following delivery, (ii) immediate drying and wrapping of the newborn, (iii) use of sterile cord clamps, (iv) breastfeeding within one hour of birth, (v) mothers’ condition assessed between 0 and 30 min after delivery, and (vi) vitamin K given to infants within 6 h of birth. The health system factors that we assessed were the level of health facility (district hospital or community health center), the cadre of the staff conducting the delivery (doctors or nurses), years of experience of staff conducting the delivery, and time of day in which care was being provided (day or night).

While all participating facilities were covered in the QI intervention and had received QI training and coaching visits, not all took up the same improvement goals. For example, some facilities worked actively on early breastfeeding while some others worked on vitamin K administration. All 12 facilities had addressed oxytocin administration at the time of the study. All facilities eventually worked on all elements of care but did so in a different order.

Twelve observers (either doctors or nurses) were recruited for the study and received one day of training on what care to observe, how to perform data entry, and instruction on ethical concerns. Each observer visited one facility over a period of three weeks in November 2014. The observers were not informed about the specific improvement aims of the facilities they were assessing, and facility staff was reporting data on all elements of care included in the study. They recorded data on elements of care during deliveries. Observers were present in the labor room for each observed delivery from admission to two hours after the delivery. Observed patients and the staff conducting the delivery were informed about the risks and benefits of participating in the study, and written consent was obtained from both. During this period of observation, the observer used a checklist to record whether or not a particular practice was done and the time when it was done. The checklist also contained information on the staff conducting the delivery, which included years of experience and professional cadre. The observer entered the data in a prepared Excel sheet that was then sent to the central project office for analysis. All hard copies of the checklists were also sent to the office after data collection.

The study was approved by the institutional review board at University Research Co., LLC. Official permission was also obtained from facility and district authorities. Written consent was obtained from all observed staff and patients. Male observers were only assigned to facilities in which male doctors already conducted deliveries. Checklists and other data collection tools did not carry any names, but used unique identifying numbers for both patients and staff; thus, precluding the possibility of identifying any patient or staff.

Statistical analysis: Data analyses were conducted using Stata/SE 13.1. We conducted univariate logistic regression of the primary outcome variables – oxytocin administration, vitamin K injection, drying and wrapping, cord care, breastfeeding, post- partum vitals assessment – on the independent variables to observe associations between the two. The independent or predictor variables included improvement goal versus not having the improvement goal, district hospital or CHC, doctors or nurses conducting delivery, years of experience of staff conducting the delivery, day or night shift on observed practice. Thereafter, we entered all independent variables into a multivariate model to conduct a stepwise backward logistic regression of each element of care as the outcome variable on the independent variables. The outcome variable "cord care" was dropped from the model as there were few cases in the cells for "not observed" (i.e., almost all cases observed performed appropriate cord care).

A total of 461 deliveries were observed, of which 362 (79%) deliveries occurred in district hospitals, 324 (70%) were attended by staff nurses, 254 (55%) of delivering clinicians had less than 5 years of experience, and 234 (51%) occurred during the day. The QI teams at these facilities had focused on improving different elements of care at the time of the study: oxytocin administration (all 12 facilities), vitamin K administration (11 facilities), early initiation of breast feeding (8 facilities), postpartum assessment (7 facilities), drying and wrapping the infant (4 facilities), and sterile cord care (4 facilities) (Table I).

TABLE I	Number of Observations Across Indicators as Improvement Aims (N=461)

Compliance with standards of care as measured by observation varied across different elements of care and between health facilities. The compliance with each element of care and the range between facilities were: sterile cord care (98%, range 53-100%), thermal care (92%, range 0-100%), vitamin K administration (91%, range 43-100%), oxytocin administration (81%, range 21-100%), initiation of early breastfeeding (63%, range 13-100%), postnatal monitoring between 0-30 minutes (34%, range 0-100%) (Table II).

TABLE II 	Proportion of Correct Care Elements Observed (N=461)

In univariate analysis, performance of the observed elements of care was higher in facilities with a team using QI methods to try to improve that element of care for cord care (100% vs 97%, P<0.05), early initiation of breast feeding (69% vs 51%, P<0.001), monitoring vital signs at 0-30 minutes (45% vs 20%, P<0.001). Drying and wrapping newborns soon after delivery was slightly higher in facilities working on the improvement aim, but the difference was not significant (95% vs 91%, P=0.15). Performance was lower for vitamin K administration (87% vs 100%, P<0.001) when facilities were trying to improve that element. All facilities chose to work on improving administration of oxytocin immediately after delivery so there was no comparison group.

In multivariate analysis, the presence of a QI team working on that specific element of care was associated with better performance of drying and wrapping infants (AOR 2.6; 95% CI 2.1, 6.6), initiation of early breast-feeding (AOR 3.6; 95% CI 2.1, 6.2), and post-partum assessment (AOR 2.7; CI: 1.7, 4.2). In the multivariate analysis, no element of care was worse when a QI team was focused on that element.

Deliveries conducted in a DH were associated with better performance of drying and wrapping infants but worse early initiation of breastfeeding, vitamin K administration, and post-partum assessment (Web Table I). When doctors conducted the delivery, women were more likely to receive post-partum monitoring but less likely to initiate early breastfeeding. Deliveries conducted by more experienced staff were associated with better performance on oxytocin administration and vitamin K administration and worse performance on early initiation of breastfeeding. Infants born during the day were more likely to receive vitamin K (Web Table I).

In this study, we observed that in the facilities where health workers were not trying to actively improve an element of care, compliance with standard care ranged from 20% for monitoring vital signs in the first 30 minutes to 100% for vitamin K administration. Compliance was significantly higher in facilities where staff were using QI approaches to improve elements of care. With the exception of vitamin K administration (where the one hospital that had not worked on this improvement aim had 100% performance), no element of care was worse when hospitals were using QI approaches to try to improve it. Apart from having a QI improvement goal, compliance was also affected by the facility and shift in which delivery took place, as well as the type of staff conducting the delivery and their years of experience. None of these factors had uniform effects; they all made some elements of care better and some worse.

This study has several limitations. First, we used observation to measure quality of care, which has the possibility of bias due to the Hawthorne effect [12]. Observers were not informed about which elements of care facilities were actively trying to improve, but it is possible that they learned this during their visits. Second, given that there were only 12 facilities in the study, we are unsure about generalizability of these findings. Given that this study was carried out in the context of an implementation effort that supported hundreds of facilities in these six states to use QI approaches to deliver better care to mothers and babies that demonstrated improvements in the same elements of care and a reduction in perinatal mortality over two years [5], we believe that the findings are generalizable at least within India. The findings from the overall program cited above also suggest that the improvement is sustainable even when facility teams move on to new improvement aims. Follow-up from one of these facilities found sustained performance on their initial improvement project more than two years after the end of the external support [13]. A third limitation is that this work only assessed a limited number of elements of care and did not measure the woman’s experience of care at all.

The lessons learned from implementing this large scale program to build QI skills in health workers and support them to use these skills to provide better care have been documented elsewhere [5,14,15]. We believe that keeping interventions simple, focusing on results, and facilitating learning improves quality of care. Each facility needs to identify the specific barriers that exist in their setting and develop solutions to these barriers. We conclude that facilities using QI approaches to ensure all women and babies receive specific elements of care provide that element of care to a greater proportion than facilities not focusing on improving that element of care.

Contributors: ES and NL: planned the study; ES: supervised data collection and led the analysis; NL, ES: wrote the manuscript and approved its final version.

Funding: The intervention described in this paper was made possible by the support of the American people through the United States Agency for International Development (USAID), through the USAID Applying Science to Strengthen and Improve Systems (ASSIST) Project, implemented by University Research Co., LLC under Cooperative Agreement Number AID-OAA-A-12-00101.

Competing Interest: None stated.