|
Indian Pediatr 2016;53: 823-828 |
|
Efficacy of a Mobile-based Application on
Quality of Care and Perinatal Mortality
|
Source Citation: Lund
S, Boas IM, Bedesa T, Fekede W, Nielsen HS, Sørensen BL. Association
between the safe delivery app and quality of care and perinatal survival
in Ethiopia: A randomized clinical trial. JAMA Pediatr. 2016;170:765-71.
Section Editor: Abhijeet Saha
|
Summary
In this cluster-randomized clinical trial in 5 rural
districts of Ethiopia, 73 healthcare facilities were randomized to the
mobile phone intervention or to standard care (control). 3601 women in
active labor were included at admission and followed-up until 7 days
after delivery to record perinatal mortality. Knowledge and skills in
neonatal resuscitation were assessed at baseline and at 6 and 12 months
after the intervention among 176 health care workers at the included
facilities. Analyses were performed based on the intention-to-treat
principle. Healthcare workers in intervention facilities received a
smartphone with the safe delivery app (SDA). The SDA is a training tool
in emergency obstetric and neonatal care that uses visual guidance in
animated videos with clinical instructions for management. The primary
outcome was perinatal death. Secondary outcomes included the knowledge
and clinical management of neonatal resuscitation (skills) of health
care workers before the intervention and after 6 and 12 months. Use of
the SDA was associated with a nonsignificant lower perinatal mortality
of 14 per 1000 births in intervention clusters compared with 23 per 1000
births in control clusters (OR 0.76; 95% CI, 0.32, 1.81). The skill
scores of intervention health care workers increased significantly
compared with those of controls at 6 months (mean difference 6.04; 95%
CI, 4.26, 7.82) and 12 months (mean difference 8.79; 95% CI, 7.14,
10.45) from baseline, corresponding to 80% and 107%, respectively, above
the control level. Knowledge scores also significantly improved in the
intervention compared with the control group at 6 months (mean
difference 1.67; 95% CI 1.02, 2.32) and at 12 months (mean difference
1.54; 95%CI, 0.98, 2.09), corresponding to 39% and 38%, respectively,
above the control level. Authors concluded that SDA was an effective
method to improve and sustain the health care workers’ knowledge and
skills in neonatal resuscitation as long as 12 months after
introduction.
Commentaries
Evidence-based Medicine Viewpoint
Relevance: The widespread availability,
acceptability, and affordability of mobile communication devices
(especially mobile phones and smart phones) have spawned a new branch of
health-care, popularly referred to as ‘mHealth’. Although there are
diverse definitions of mHealth [1], the underlying common theme is the
utilization ofmobile and wireless devices to deliver some form of
health-care. The World Health Organization (WHO) also recognizes mHealth
as a separate division of eHealth [2] with the potential to enhance
health-care delivery in developing countries. It is therefore not
surprising that scientific literature is replete with studies describing
the benefits of utilizing one or other form of mHealth to enhance
health-care services. There are also a limited number of well-designed
randomized controlled trials and systematic reviews addressing whether
pregnancy and childbirth outcomes can be enhanced through mHealth. In a
previous studies led by Lund [3,4], pregnant women were randomized to
receive a mHealth intervention or standard care. The intervention
included educational input, visit reminders and emergency call facility
through a combination of text messages and two-way voice communication.
The odds of still birth and perinatal mortality were lower in the
intervention group, although only the latter achieved statistical
significance. However, there was no difference in mortality among the
infants during the first six weeks of life. In contrast, a case control
study in Nigeria did not find any significant improvement in maternal
mortality among pregnant women who were provided mobile phones for
communicating with health-care workers [5]. Two recent systematic
reviews [6,7] evaluating the impact of various mHealth interventions on
maternal and/or newborn health-care outcomes reported diverse benefits
in terms of educational empowerment, enhanced confidence and some
surrogate markers of morbidity. However these reviews did not identify
additional studies reporting hard outcomes such as mortality. The recent
study by Lund, et al, [8] is a cluster-randomized controlled
trial comparing the efficacy of a mobile phone based application called
Safe Delivery App (SDA), on perinatal mortality as well as health-care
workforce knowledge and skills in newborn resuscitation. Table
I presents a brief summary of the trial.
Table I Brief Summary of the Trial [8]
Objective |
To compare the efficacy of health-care workers’ utilization of
Safe Delivery App (SDA) on perinatal mortality, as well as
health workers’ knowledge and skills in newborn resuscitation. |
Study design |
Cluster randomized trial
|
Study setting |
70 health-care facilities in in 5 Ethiopian rural districts. |
Study duration
|
September 2013 to January 2015 (17 months) |
Population (P) |
Inclusion criteria: Babies born to 3600 pregnant women enrolled
during the study period; and 176 health-care workers with basic
maternal/newborn skills posted in the 70 facilities, comprised
the study population.
|
Intervention (I) |
Health-care workers in this group received training (over one
day) to use the Safe Delivery App which provides guidance in
newborn resuscitation knowledge and skills at the point-of-care;
along with a catalogue of essential medicines and equipment.
|
Comparison (C) |
Health-care workers in this group did not receive additional
training.
|
Outcomes (O) |
Primary outcome: Perinatal mortality rate Secondary outcomes:
Knowledge and skills of health-care workers in newborn
resuscitation at 6 and 12 months after initiating the
intervention, measured through a scoring system. |
Sample size |
A priori sample size calculation necessitated randomization of
70 health-care facilities to demonstrate 40% decline in the
primary outcome with alpha and beta of 5% and 20% respectively.
|
Similarity of groups at baseline |
Pregnant women in the two groups were comparable with respect to
socio-economic status, literacy level, age, occupation, parity,
and place of delivery.In general, it appears that the
health-care workers in the intervention group were less
experienced (in terms of number of deliveries conducted, and
prior experience with smart phones). However, the baseline
knowledge and skill with respect to newborn resuscitation were
comparable in both groups.
|
Randomization |
Computer generated sequence was used to randomize health-care
facilities to the two groups, stratified by district as well as
level of care. |
Allocation concealment |
Not mentioned
|
Blinding |
Participants, health-care workers, and outcome assessors (i.e
those who scored the knowledge and skills of health-care
workers) were not blinded.
|
Selective outcome reporting |
All relevant measures related to the stated outcomes were
reported.Please see additional notes.
|
Incomplete outcome reporting |
Of 73 facilities randomized, 70 were included in the analysis.
Of 3601 pregnant women randomized, 482 (13.4%) were not included
in the analysis (no reason mentioned). This included 9.2% of
women in the intervention group and 16.7% in the control
group.Of 176 health-care workers randomized, 25% in the
intervention group and 27% in the control group were not
available for analysis. |
Statistical tests |
Appropriate. |
Main results
|
Intervention vs Control group |
|
• Perinatal mortality rate: 14/1000 births vs 23/1000 births (OR
0.76, 95% CI 0.32, 1.81) |
|
• Skill score at 6 months: Mean difference 6.0 (95% CI 4.3, 7.8) |
|
• Skill score at 12 months: Mean difference 8.8 (95% CI 7.1,
10.5) |
|
• Knowledge score at 6 months: Mean difference 1.7 (95% CI 1.0,
2.3) |
|
• Knowledge score at 12 months: Mean difference 1.5 (95% CI 1.0,
2.1) |
Critical appraisal: This trial had several
strengths. An appropriate study design was chosen and the participants
(pregnant women as well as health-care workers) were cluster randomized
rather than individually randomized. This had the advantage of
preventing cross contamination between the intervention and control
groups. A priori sample size calculation was made and met. The baseline
characteristics of the population suggest that any interventions to
empower the community would be welcome.
However, there are some important limitations not
addressed by the investigators. Although an appropriate randomization
procedure was used, allocation concealment (an important component in
assessing risk-of-bias in trials) [9] has not been addressed at all.
Further, it is understandable that blinding of participants (health-care
workers as well as the pregnant women being served) for the primary
outcome was not feasible in this trial; however blinded independent
outcome assessors could have assessed knowledge and skills of the
health-care workers (secondary outcomes) rather than local supervisors.
In terms of participant attrition, about 13% of the enrolled pregnant
women were unavailable for analysis (although no specific reasons are
ascribed); the distribution in the two groups was also quite different
(see Table I). More important, over 25% of the
health-workers in both groups also dropped out and were unavailable at
analysis. It appears that these workers moved from their posts but it is
unclear if these transfers vitiated the randomization procedure.
Likewise although most of the important outcomes were considered, it
would have been helpful to learn additional secondary outcomes viz (i)
number of babies resuscitated in each group, and (ii) causes of
perinatal death where it could be attributed. These two outcomes would
help to assess the potential impact of resuscitation (as a life-saving
measure) in this study. These limitations create a moderate to high risk
of bias.
Table II Selected Indicators Reflecting Child Survival and the Healthcare Delivery System in Ethiopia [11-15]
Year |
U5MR |
IMR |
NMR |
Reference |
2009 |
104 |
67 |
36 |
11 |
2010 |
106 |
68 |
35 |
12 |
2011 |
77 |
52 |
31 |
13 |
2012 |
68 |
47 |
29 |
14 |
2013 |
64 |
44 |
28 |
15 |
IMR: infant mortality rate, NMR: neonatal mortality rate,
U5MR: Under five mortality rate |
In the public domain, there is no readily available
source of perinatal mortality data from the participating Ethiopian
districts. However, other health indicators for Ethiopia before and
during the study period (Table II) suggest that the
country has witnessed a dramatic and progressive improvement in overall
child survival in the most recent years, although previous years did not
reflect this pattern. Based on these data, it is possible that the lower
(than expected) early neonatal mortality rate observed in this study
could reflect overall improvements in health-care services. It is also
possible that that this could reflect the Hawthorne effect [10], whereby
mere inclusion in a trial (with a performance observation component)
could have improved the professional behavior of the health-care
workers. These two possibilities could also account for the absence of a
statistically significant different difference in the primary outcome
between the two groups. The authors did not explore these possibilities
further.
Perhaps the most important flaw in this study lies in
the authors’ interpretation of their findings. They attributed all the
results to the Safe Delivery App. Careful perusal of the article [8]
shows that the intervention group received one day training in the app
contents and usage; whereas the control group did not receive any
training. Therefore the differences in the two observed groups is likely
a result of the additional training (in this case, with the app), rather
than the app itself. This distinction is more than semantic because (i)
it suggests training enhances performance, and (ii) the
appropriate study design to compare the efficacy of the app would
require the control group to be administered the same training (level,
content, and duration) through some means other than the app. Otherwise
there is the distinct possibility that providing health-care workers the
contents of the app, in another readily accessible format (such as flip
boards, charts, etc) could yield similar results. In other words, this
trial reflects the benefit of refresher training and availability of
resources at the point-of-care, rather than the advantage of a mHealth
platform.
The second issue open to interpretation is whether
the statistically significant differences in health workers’ knowledge
and skills (both in the intra-group as well as inter-group comparisons)
have any clinical relevance. For example, out of 24 attainable points in
the knowledge domain the app group only achieved a mean score of 13 (at
6 months) and 16 (at 12 months). Although these were statistically
significant increases, they demonstrate a knowledge deficit despite
providing refresher training and a ready-to-use resource at the
point-of-care. Even less impressive was the increase in the skills
domain wherein out of 12 attainable points, the app group showed
increase in score from 4 at baseline to 6 (at 6 months) and 5.5 (at 12
months). This again suggests a potential clinically important skill
deficit. Continuing along this line of thought, the statistically
significant difference between the scores of the two groups may not have
clinical relevance when the workers actually face a newborn requiring
resuscitation. For this reason, it is especially important to know how
many babies were actually resuscitated in each group and to establish
the number of deaths due to inadequate/inappropriate resuscitation. This
point has been highlighted above also.
One issue that is unclear is the authors’ claim of
performing intention-to-treat (ITT) analysis. Since only available
participants (health-care workers as well as pregnant women) were
included in the analysis, as opposed to all those randomized, this is
not ITT analysis. Even if health-care facilities are considered, 70 of
73 randomized were included in the analysis. A minor point (probably
typographical) is that although there were 1645 pregnant women in the
control group, Figure 1 states 1665.
Extendibility: The SDA is a free-to-access
resource that is broadly applicable to most resource-constrained
settings. It has the advantage of being usable at the point-of-care
without the need to go online to access its content. In that sense, it
serves as an interactive ‘ready reckoner’ for peripheral health-care
workers. This makes it usable in diverse health-care settings especially
where mobile phone penetration is high (such as India). Previous studies
(and common sense) confirm that education, motivation and refresher
training programs for health-care workforce (and also health-care
consumers) enhance the delivery as well as acceptance and utilization of
health-care services. Limited data from developing countries [16,17]
including India [18] report that mobile phones could be useful to
provide such inputs.
Conclusion: This RCT suggests (but does not
prove) that enhancing the training of health-care workers in newborn
resuscitation (through the usage and application of a mobile phone based
app) could potentially improve newborn survival and empower the health
workers’ knowledge and skills.
Funding: None; Competing interest: None
stated.
Joseph L Mathew
Department of Pediatrics,
PGIMER, Chandigarh, India.
Email:
[email protected]
References
1. Healthcare Information and Management Systems
Society (HIMSS). Definitions of mHealth. Available from:
http://www.himss.org/definitions-mhealth. Accessed August 13, 2016.
2. World Health Organization. mHealth: New Horizons
for Health Through Mobile Technologies. Available from:
http://www.who.int/goe/publications/goe_mhealth_web. pdf. Accessed
August 13, 2016
3. Lund S, Rasch V, Hemed M, Boas IM, Said A, Said K,
et al. Mobile phone intervention reduces perinatal mortality in
Zanzibar: secondary outcomes of a cluster randomized controlled trial.
JMIR Mhealth Uhealth. 2014;2:e15.
4. Lund S, Hemed M, Nielsen BB, Said A, Said K,
Makungu MH, et al. Mobile phones as a health communication tool
to improve skilled attendance at delivery in Zanzibar: a cluster–randomised
controlled trial. BJOG. 2012;119:1256-64.
5. Oyeyemi SO, Wynn R. Giving cell phones to pregnant
women and improving services may increase primary health facility
utilization: a case–control study of a Nigerian project. Reprod Health.
2014;11:8
6. Lee SH, Nurmatov UB, Nwaru BI, Mukherjee M, Grant
L, Pagliari C. Effectiveness of mHealth interventions for maternal,
newborn and child health in low- and middle-income countries: Systematic
review and meta-analysis. J Glob Health. 2016;6:010401.
7. Sondaal SF, Browne JL, Amoakoh-Coleman M,
Borgstein A, Miltenburg AS, Verwijs M, et al. Assessing the
effect of mHealth interventions in improving maternal and neonatal care
in low- and middle-income countries: A systematic review. PLoS One.
2016;11:e0154664.
8. Lund S, Boas IM, Bedesa T, Fekede W, Nielsen HS,
Sørensen BL. Association between the safe delivery app and quality of
care and perinatal survival in Ethiopia: A randomized clinical trial.
JAMA Pediatr. 2016;170: 765-71.
9. No authors listed. The Cochrane Collaboration’s
Tool for Assessing Risk of Bias. Available from:
http://ohg.cochrane.org/sites/ohg.cochrane.org/files/uploads/Risk%20of%20bias%20assessment%20tool.pdf.
Accessed August 15, 2016.
10. No authors listed. Hawthorne Effect. Available
from: https://en.wikipedia.org/wiki/Hawthorne_effect. Accessed August
15, 2016.
11. UNICEF. State of the World’s Children. 2011
Report. Available from:
http://www.unicef.org/sowc2011/pdfs/SOWC-2011-Main-Report_EN_02092011.pdf.
Accessed August 15, 2016.
12. UNICEF. State of the World’s Children. 2012
Report. Available from:
http://www.unicef.org/sowc2012/pdfs/SOWC%202012-Main%20Report_EN_13Mar2012.pdf.
Accessed June 15, 2016.
13. UNICEF. State of the World’s Children. 2013
Report. Available from: http://www.unicef.org/sowc2013/
files/SWCR2013_ENG_Lo_res_24_Apr_2013.pdf. Accessed June 15, 2016.
14. UNICEF. State of the World’s Children. 2014
Report. Available from:
http://www.unicef.org/sowc2014/numbers/documents/english/SOWC2014_In%20
Numbers_28%20Jan.pdf. Accessed June 15, 2016.
15. UNICEF. State of the World’s Children. 2015
Report. Available from:
http://www.unicef.org/publications/files/SOWC_2015_Summary_and_Tables.pdf.
Accessed June 15, 2016.
16. Cormick G, Kim NA, Rodgers A, Gibbons L, Buekens
PM, Belizán JM, et al. Interest of pregnant women in the use of
SMS (short message service) text messages for the improvement of
perinatal and postnatal care. Reprod Health. 2012; 9:9.
17. Zurovac D, Sudoi RK, Akhwale WS, Ndiritu M, Hamer
DH, Rowe AK, et al. The effect of mobile phone text-message
reminders on Kenyan health workers’ adherence to malaria treatment
guidelines: a cluster randomised trial. Lancet. 2011;378:795-803.
18. Datta SS, Ranganathan P, Sivakumar KS. A study to
assess the feasibility of Text Messaging Service in delivering maternal
and child healthcare messages in a rural area of Tamil Nadu, India.
Australas Med J. 2014;7:175-80.
Neonatologist’s Viewpoint
There is an increased interest in improving health
using mobile phones. While the previous decade had novel methods based
on short messages delivery to basic phones [1], recently there has been
a fast paced development in app-based smartphone technology. Most health
apps focus on nutrition and fitness [2], but some new apps have been
developed for healthcare delivery and education.
This app ecosystem related to maternal and newborn
care has been fairly limited to training, diagnostic algorithms,
heart-rate monitoring, eye photography, improvement of intubation
practices, etc. A recent systematic review of mHealth research in
developing countries listed out the poor methodological qualities of
studies that included randomized control trials as well [3].
JAMA Pediatrics reports a well-conducted study
that demonstrates significant improvement in skills over a 12-month
period, along with a modest improvement in knowledge as well [4]. The
app requires an active internet connection only for the initial
download. It has a simple interface and does not hang during usage and
also has sections on videos, action cards, drugs, procedures and
questionnaires related to maternal management (five areas), newborn
resuscitation and newborn management (five areas). In order to hold the
interest of users, the app uses push messaging and automatically
assesses the app holder every three months on his/her confidence on
various areas of BEmONC (Basic Emergency Obstetric and Newborn Care).
The key feature questionnaire has 15 cases and 55 questions. If a user
selects a harmful option, half the score is deducted, and the entire
score for the question is deducted if a critically harmful option is
chosen. Thus the user is taught to avoid serious mistakes and he/she can
take a self-assessment questionnaire at any point of time. This will
allow reflective learning, which is one of the core features of adult
learning.
The app allows the user to practice at will and hone
skills, and also allows for the same user to check back on correct
processes and/or procedures after attending to or before attending to an
emergency. As a ready reference, it allows the user to continuously
improve skills and techniques. Low dose high frequency simulation works
on the same principle and has been shown to improve and retain skills
over a long period of time [5].
Application of this app to countries like India has
several challenges. Internet and smartphone penetration varies across
the country, as do the number of languages that need to be used. We also
have various other programs in the public sector that apply to mothers
and newborns, which are not captured in the app. Thus it cannot be
easily applied to the current system. However, with language changes, it
can be used in small facilities to promote good skills and ideal
practice, and deliver newborns with chances of better outcomes to a
neonatologist for further management. The authors have not reported it,
but with such app technologies it is also possible to collect user data.
It is also possible for the app to provide information on the number of
times the app was accessed and used. GIS tagging can also allow the
supervisors to locate place of usage, which can be correlated to actual
delivery data and determine usage on a practical scale. These points can
then be utilized by supervisors to track individual users and assess
competencies appropriately. Many facilities in India are solitary and
not interconnected. The app can be used as a complementary tool to
improve retention of skills at these centers.
In conclusion, appropriate implementation of this app
will improve the outcomes of neonates that the neonatologists care for
by picking them up early and facilitating better management.
Funding: None; Competing interest: None
stated.
Somashekhar Marutirao Nimbalkar
Depertment of Pediatrics,
Pramukhswami Medical College,
Anand, Gujarat, India.
Email: [email protected]
References
1. Hall AK, Cole-Lewis H, Bernhardt JM. Mobile text
messaging for health: a systematic review of reviews. Annu Rev of Public
health. 2015;36:393-415.
2. Higgins JP. Smartphone Applications for Patients’
Health and Fitness. Am J Med. 2016;129:11-9.
3. Lee SH, Nurmatov UB, Nwaru BI, Mukherjee M, Grant
L, Pagliari C. Effectiveness of mHealth interventions for maternal,
newborn and child health in low- and middle-income countries: Systematic
review and meta-analysis. J Global Health. 2016;6:010401.
4. Lund S, Boas IM, Bedesa T, Fekede W, Nielsen HS,
Sorensen BL. Association between the safe delivery app and quality of
care and perinatal survival in Ethiopia: A randomized clinical trial.
JAMA Pediatr. 2016;170: 765-71.
5. Sutton RM, Niles D, Meaney PA, Aplenc R, French B,
Abella BS, et al. Low-dose, high-frequency CPR training improves
skill retention of in-hospital pediatric providers. Pediatrics.
2011;128:e145-51.
Public Health Expert’s Viewpoint
In October 2015, India reached a milestone of 1
billion of mobile phone subscriber base [1]. With this increasing
proliferation of mobile phones in the country, there is a potential for
its utilization in delivering public health services. Many innovations
are exploring its use in low and middle income countries (LMIC) [2].
Though Lund, et al. [3] report a significant improvement
in skills and knowledge of neonatal care at 6 and 12 months, they did
not find a statistically significant reduction in perinatal mortality.
This failure to find an association with public health outcome should
not discourage wider adaptation of mHealth application for training of
health workers. In LMIC settings achieving skilled birth attendance is a
moving target. Getting community health workers to training sessions
often happens at the cost of their already overloaded work schedules.
Therefore, innovative tele-training sessions are the need of the day.
Conversion of the increased knowledge and skill on neonatal care into
reduced perinatal mortality requires system wide changes including
improved quality of care through the maternal and child health continuum
starting from care of adolescent girls, antenatal mothers, parturient
women, through to postnatal care and care of the child. Future studies
of such mobile applications should also try and measure change in
practices of the health care providers following the training, as we
know that knowledge and skill do not always translate to practices.
Funding: None; Competing interest: None
stated.
Vijayaprasad Gopichandran
Department of Community Medicine,
ESIC Medical College & PGIMSR,
Chennai, India.
Email:
[email protected]
References
1. Saritha R. India just crossed 1 billion mobile
subscribers milestone and the excitement’s just beginning. Available
from:
http://www.forbes.com/sites/saritharai/2016/01/06/india-just-crossed-1-billion-mobile-subscribers-milestone
-and-the-excitements-just-beginning/#5d7312895ac2. Accessed July 18,
2016.
2. Hall CS, Fottrell E, Wilkinson S, Byass P.
Assessing the impact of mHealth interventions in low- and middle-income
countries – what has been shown to work? Glob Health Action.
2014;7:256-06.
3. Lund S, Boas IM, Bedesa T, Fekede W, Nielsen HS,
Sorensen BL. Association between the safe delivery app and quality of
care and perinatal survival in Ethiopia: A randomized clinical trial.
JAMA Pediatr. 2016;170: 765-71.
|
|
|
|