Evaluating Knowledge and Skills Retention of Helping Babies Breathe Training among Trainees over the Period of Time in Kavre, Nepal

Introduction: Helping Babies Breathe and Helping Babies Survive (HBS) programs are designed to develop basic neonatal resuscitation skills and knowledge aimed at the first minutes after birth. The programs were implemented at Scheer Memorial Adventist Hospital in Banepa, Kavre, Nepal. This study aimed to evaluate the success of training participants and maintaining their skills over time. Methods: The programs were implemented at SMAH over three years for staff and students involved in maternal / foetal medicine. Skills and knowledge of 119 participants were evaluated prior to training, after training, and again at five separate time intervals using standardised tests. Results: The training resulted in increased scores on knowledge testing from pre-test to post-test. Knowledge retention over time was shown to have a statistically significant decrease from post-test to retest. An objective structured clinical examination A performance did not show statistically significant decrease over time from post-test to retest. For the objective structured clinical examination B, there was a statistically significant decrease in scores over time from post-test to retest. Conclusions: The training resulted in a significant improvement in passing scores on knowledge and skills testing. While a statistically significant decrease in scores on testing was seen, the clinical significance of these decreases is questionable. This study shows that with implementation and continued practice, both knowledge and skills for basic neonatal resuscitation can be achieved and maintained.


INTRODUCTION
Globally approximately 7000 newborn deaths occur daily, with nearly one-third of these occurring within the first day of life. 1 At least 660,000 deaths occur annually as a result of intrapartum complications, specifically birth asphyxia. 2 In Nepal, the current neonatal mortality rate is 21 per 1000 live births. 3 The use of basic neonatal resuscitation has been estimated to decrease neonatal mortality by nearly one third. 4 However, the use of resuscitation guidelines is not adequate in Nepal. 5 Helping Babies Breathe (HBB) focused its efforts on the first minute of life, as this is the ideal threshold for initiating ventilation to successfully resuscitate an infant. 6 A known challenge with the Helping Babies Survive (HBS) program is the long-term retention of knowledge and skills by trainees. This study sought to evaluate the long-term retention of skills and knowledge at Scheer Memorial Adventist Hospital (SMAH) within Kavre District, Nepal after the implementation of HBB / HBS in fall of 2017.

METHODS
SMAH, in Kavre district, Nepal, was the initial site for HBB training. The program was implemented in August 2017, at SMAH through a strategic partnership with Centura Global Health Initiatives (GHI), Denver, Colorado, USA, with the support of the local Kavre District Health Department. Only data from SMAH was used for the purposes of this study, though the HBB team has expanded training to surrounding health posts. The HBS program introduced by GHI provides a framework of activity and training, over a three-year period. Experienced leaders in maternal-child health are identified and provided salary support to champion the HBS program locally. Their responsibilities include monitoring and evaluating the skills of birth attendants at both SMAH and outlying health posts, organising continued training and management of the necessary equipment for teaching and resuscitating infants at the time of delivery. In addition, the in-country champion collects data on the number of hours spent at each practice station within the district and, with the Government's permission, collects birth logs from participating health facilities . The ultimate success of the  program relies on the investment of local leaders to  support the training and provide the human  resources to build a sustainable program, as  evidenced by the support provided by SMAH. HBB training takes place over two days. This involves role-playing, going through resuscitation algorithms, and discussions of questions suggested by HBB curriculum. Initial training started with key nursing and physician providers as selected by the obstetrics (OB) nurse managers. The HBB curriculum was implemented per American Academy of Paediatrics (AAP) recommendations. During the training, individuals are given the opportunity to practice skills with close supervision. A subset of these individuals were further trained to be Master Trainers per the HBB protocol, and assumed the responsibility of continued training, with US Master trainers providing mentoring and support. The College of Nursing (CoN) has also been added to the trainees. All students in the CoN now undergo HBB training in the standardised HBB training protocol. HBB training was provided to 119 individuals that included nursing students, auxiliary nurse midwife (ANM), certified medical assistants (CMA), proficiency certificate level nurses (PCL), bachelor level nurses, physicians, master nurses, and administrators. Although training continued until COVID-19 pandemic restrictions were mandated, for the purposes of this study, individuals trained between 8 / 2017 through 10 / 2019 were used for re-testing.
A knowledge-check, multiple choice questionnaires is given to individuals undergoing the HBB training on the first day of the course before teaching begins and then is repeated at the end of the course. A passing score on the knowledge check was 15/18. The Objective Structured Clinical Exams (OSCE) was developed by the AAP and others to evaluate resuscitation skills based on the HBB curriculum. These are given by Master Trainers and evaluate the ability of an individual to correctly assess the need for resuscitation interventions based on a clinical scenario. Action items are then scored as being done or not done. Scores on initial knowledge checks and OSCE A and OSCE B were recorded and maintained in an Excel document. Practice stations were created in the obstetrical unit to allow providers and students who most frequently provide resuscitation the opportunity to practice as time allowed. A practice log was maintained. The practice station included a mannequin, suction, bag-mask, a visual action plan, and other supplies used in the training.
To evaluate the retention of knowledge and skills, repeat OSCE and knowledge checks were conducted from November 2019 through February 2020. These occurred in groups of learners with Master trainers precepting the skill checks. Each learner was re-tested once with the time since the initial training recorded. Knowledge checks were completed and graded in the same session. For the skills testing, the OSCE was done. Prior to the retesting some coaching and practice was allowed to help students remember the skills they had learned.
Data analysis was done using SPSS 20 statistical software. A paired t-test was chosen to compare means between pretest, post-test and retest data. Ethical approval was obtained from the Institutional Review Committee at SMAH.
The knowledge and skill retention among participants of the HBS training program was analysed at the 0.05 level of significance using the paired sample statistical test (See Table 1

DISCUSSION
The HBS program implemented at SMAH was successful at increasing knowledge among participants as noted by a significant increase in test scores at baseline and after the initial training (p < 0.05). Further, looking at the post-test scores to re-test scores, we can see that this knowledge is retained. Knowledge and skill retention over time have been demonstrated in other studies as well. Specifically, it was found that skills and knowledge were retained with a robust practice program in place. 7 In our study there were differences over time in post-test to retest scores. These differences are reflected more in variability of scores, rather than in mean scores, suggesting that most participants had reasonable knowledge retention over time. Skill retention followed a similar pattern with OSCE A scores staying stable over time. OSCE B, which is a somewhat more complicated set of skills, did show a statistically significant decline over time, but again reflected more in the variability of scores rather than in the means. When looking at the knowledge check and the skills testing, the differences in the mean from post-test to retest show a difference of less than what amounts to a single question on the knowledge check or a single check box for skills testing. Nonetheless, this difference is worth exploring, as it may provide insight into better practices for longterm skill retention. These statistically significant differences are also reflected in the ANOVA analysis, which shows that there is difference between the various re-test time intervals for the knowledge test and OSCE B. Interestingly, when looking at the mean scores at these time points, it appears that they are quite variable with dips at 6 -12 months and again at 18 -24 months, but not at 24 -30 months. This variability is important to note as it may be due to several factors. First, scores may have been impacted by differences in testing methodology rather than distinct differences in skill retention. Second, it is important to note that there is a wide spectrum of participants, varying by education level. This is particularly important to note as these differences did reach statistical significance in analysis and are likely to impact the skew of results over time depending on the educational makeup of the group during retest at a specific time point. Another study evidenced decline in skills over time, 8 specifically found in relation to skills testing through the OSCE B, which is similar to our findings here. ANOVA analysis looking at various education levels is also worth noting, as it would seem that overall, the highest scores are at either end of the educational spectrum with students and Masters level educated groups having higher means overall.
The variability across time points is notable and may reflect a limitation of this study. Having different assessors, which is necessary for efficiency and practicality, may lead to some differences in scores. This is most relevant for the skills testing as there can be some subjective differences in assessing skills performed versus not performed. As noted above, the variation in education level and the demographic breakdown of who was tested at varying time points may have impacted results as well. Additionally, the degree to which students had the opportunity to practice prior to the test may have impacted the retesting scores for knowledge and skills. However, given the overall maintenance of knowledge and skills over time, this may also suggest that with simple practice opportunities these skills and knowledge are able to be refreshed quickly. Overall, the data indicates that this program is successful in the development and retention of life saving skills and knowledge for neonatal resuscitation. Based on this study it seems feasible to use regular refresher practice to maintain these skills. More data would be helpful in developing specific guidelines on the frequency and depth of practice needed to maintain skills and may be an area of future research.

CONCLUSIONS
Overall, this study demonstrates that not only is the HBS program successful in improving knowledge in its initial training, but that the skills and knowledge developed are retained over time when sufficient support and dedicated staff are tasked with managing the continued training for new employees, continued practice and skill retention for established employees, and building a sustainable program with proper equipment available for every birth. This is important as HBS is designed for implementation in areas where continued involvement can be challenging and long-term involvement by volunteers from the US and other countries cannot be guaranteed indefinitely, as we have seen with the recent COVID-19 pandemic. The data from this study suggests that with required practice and easily accessible practice stations, the maintenance of skills and knowledge can be sustained. Programs like this one are relevant for global health to ensure continued improvement in neonatal mortality. Further studies are underway to determine the impact this program has had on neonatal mortality.