Evaluation of the Mental Health First Aid program in undergraduate nursing students

Background: The Mental Health First Aid (MHFA®) educational program has the potential to increase mental health literacy, reduce personal stigma, and empower individuals for their mental health. Objective: To evaluate the impact of MHFA training program in undergraduate nursing students. Methodology: Pre-experimental study, with a single group design, preand post-intervention assessment, applied to a sample of 46 undergraduate nursing students. The Questionnaire for Assessment of Mental Health Literacy was applied as a data collection tool. The data were analyzed using the IBM SPSS V24.0 and G-Power 3.1 software programs. Results: At the end of the program, students showed an improvement in all components of mental health literacy about depression, increasing their confidence to provide first aid. Conclusion: The program proves to be an adequate tool for increasing the mental health literacy of nursing students.


Introduction
The Primeiros Socorros em Saúde Mental (PSSM) program, translated from the original Mental Health First Aid Educational Program (MHFA®), was created and developed in Australia, at the beginning of this century, by Anthony Jorm and Betty Kitchener . It is a program based on the concept of mental health literacy (MHL), as coined by Anthony Jorm and collaborators (Jorm et al., 1997). When defining the MHL rate as the constellation of beliefs and knowledge about mental health problems (including disturbances), which allows recognizing, managing, and preventing them in the daily life (Jorm, 2000(Jorm, , 2014, the author points to a knowledge directed at action in favor of the person's own mental health and those around him/her. The concept of MHL operates in five components, in particular: (a) recognition of mental problems and disorders to promote and facilitate the seeking of help; (b) knowledge about professionals and treatments available; (c) knowledge about effective self-help strategies; (d) knowledge and skills to provide first aid and help to others; (e) knowledge about methods of mental disorder prevention (Jorm, 2014). The MHFA program is the corollary of the work developed around the concept of MHL and its operationalization, with emphasis on the component of knowledge and skills to provide first aid and help to others, that is, the action in favor of mental health. The MHFA is the initial aid provided to someone who is experiencing a mental-health-related crisis or developing a mental health problem. Help is given until the crisis is resolved or the person receives the appropriate professional treatment Loureiro, 2014). The MHFA program follows multiple typologies and has been applied in the most diverse cultural, social, and economic realities and contexts, preserving the principles, the modular structure, and the plan of action, all the while respecting the contextual and situational characteristics of target groups to which it applies (Jorm, 2014). The objective of the program is to teach people to recognize the signs and to appreciate the symptoms of problems and mental disorders while indicating the most appropriate way to provide first aid and initial aid, whether the person is experiencing a mental-health-related crisis or even developing the disease. In this sense, it seeks to promote the call for aid and the referral to adequate professionals and other support means specialized in the field of mental health Loureiro, 2014;Loureiro, Sousa, & Gomes, 2014). Like the first-aid courses, which include a mnemonic associated to procedures/intervention actions, this program also has a plan of action, associated to a mnemonic which we translated for ANIPI (originally, ALGEE). Each letter ( Figure 1) designates an action (Loureiro, 2014;Loureiro et al., 2014), which consists of: A -Assess the risk of suicide or harm; N -Listen non-judgmentally; I -Give reassurance and information; P -Encourage the person to get appropriate professional help; I -Encourage self-help strategies. means specialized in the field of mental health Loureiro, 2014;Loureiro, Sousa, & Gomes, 2014).
Like the first-aid courses, which include a mnemonic associated to procedures/intervention actions, this program also has a plan of action, associated to a mnemonic which we translated for ANIPI (originally, ALGEE). Each letter ( Figure 1) designates an action (Loureiro, 2014;Loureiro et al., 2014), which consists of: A -Assess the risk of suicide or harm; N -Listen non-judgmentally; I -Give reassurance and information; P -Encourage the person to get appropriate professional help; I -Encourage self-help strategies. The implementation of the MHFA program to higher education students, particularly of The implementation of the MHFA program to higher education students, particularly of health courses, as is the case of nursing, is based on evidence of the substantial impact that various stressors may have on the health and well-being of students, with an emphasis on the academic requirements (workload, study-related problems and difficulties), challenges related to clinical teaching (fear of unknown situations, possible clinical error, or even the handling of technical equipment), and the repeated contact with the suffering and the dying process of patients (Pulido-Martos, Augusto-Landa, & Lopez-Zafra, 2012). At the same time, higher education students manifest a very modest level of MHL (McCann, Lu, & Berryman, 2009), which often contributes to the worsening of health situations (Loureiro, 2014). This research aims to assess the impact of the MHFA program on the mental health literacy of nursing undergraduate students. Given that the MHFA program includes different types of mental health problems (e.g., anxiety, psychosis), we decided to focus this assessment only on depression.

Background
The scientific evidence, produced in accordance with national and international proceedings, concerning the assessment of the MHFA program impact on nursing students, including pilot studies, is limited to three studies, one national (Sousa, 2015) and two international (Bond, Jorm, Kitchener, & Reavley, 2015;Burns et al., 2017). However, there are three literature systematic reviews, one without meta-analysis (Kitchener & Jorm, 2006) and two with meta-analysis (Hadlaczky, Hökby, Mkrtchian, Carli, & Wasserman, 2014;Morgan, Ross, & Reavley, 2018) about the impact of the program in different contexts and populations. Regarding the implementation of the program in samples of nursing students, we point out the study conducted in the Portuguese context by Sousa (2015). It was a pilot study which implemented the MHFA program in a sample (n) of 16 students, 87.50% female and 12.50% male, recent nursing graduates (a week after the conclusion of the course), with an average age of 21.86 years (SD = 0.54 years). At the end of the program, the correct identification of depression was 62.50%. It should be pointed out that 81.30% of the participants say they would seek help in a situation similar to that described in the vignette. As regards the trust in providing aid after completing the program, the mean was 3.50 (SD = 0.65 points) in a scale that ranged from 1 (not confident) to 5 (extremely confident). Relating to international studies, we highlight the work of Bond et al. (2015), carried out in Australia with nursing (n = 292) and medicine (n = 142) students. The employed design, with evaluation of pre-and post-test, used two different interventions, one with face-to-face training and another with online training. The program was applied to four subgroups. Nursing students were divided into two subgroups, one subgroup received the online program and the other the face-to-face program, and the same thing happened with medical students. As regards the nursing students, the program revealed high effect sizes (Cohen's d; in both course types) for knowledge, intention to provide first aid, and confidence in providing help and support. Concerning the level of (personal and perceived) stigma, the observed effect sizes were much more modest, lying between the low and moderate. The study of Burns et al. (2017), also conducted in Australia, used a sample of students in the 1 st year of the nursing course, with an experimental design, including an experimental group (n = 59) and a control group (n = 81), and three assessments, pre-, post-intervention and follow-up (2 months after the intervention).
The study results showed a statistically significant improvement in knowledge, confidence to help someone, intention to provide help, and a reduction in the social stigma and distance. The authors present the explained variation measures (partial eta-squared) to determine the effect sizes. The results, although not statistically significant in 60% of the statistical tests (ANOVA for repeated measures: group*time), showed modest (moderate) values of explained variation. As regards systematic literature reviews with meta-analysis, the works include a substantial number of studies applied to groups other than higher education students.

Evaluation of the Mental Health First Aid program in undergraduate nursing students
The first review performed by Hadlaczky et al. (2014) analyzed 15 primary studies. As mentioned above, the studies relate to differentiated populations, but the comparative analyses showed an increase of knowledge about mental health problems, a decrease of stigmatizing attitudes, and an increase of behaviors of intention to provide first aid to the person in a suffering situation. More recently, the review by Morgan et al. (2018), based on 18 primary studies, points out that effect sizes range between low and moderate, at the end of the program, and up to 6 months after their completion. The improvements occurred in knowledge about mental health first aid, with low to moderate values of the effect size (Cohen's d) measure in the belief about the treatment efficacy, confidence in providing first aid to people with mental health problems, and intention to provide first aid. In terms of reducing stigma, the effect size observed was reduced. All the mentioned studies, whether individually analyzed, either based on the reading and analysis of systematic reviews, point out the effectiveness of the PSSM program in empowering people for the provision of first aid and mental health first aid, thus contributing substantially to the improvement of mental health literacy about depression.

Hypothesis
The attendance of the MHFA program by nursing students contributes to the increase in mental health literacy about depression.

Methodology
It is a quantitative, level IV, and pre-experimental study, with a single group and preand post-intervention evaluation (Campbell & Stanley, 2005). The study sample was selected from the population of enrolled students attending the second, third, and fourth school years of Bachelor's Degree in Nursing (CLE) of a teaching institution of the central region of Continental Portugal. From the total population, 90 students were randomly selected, divided equally between the school years, and were contacted via e-mail to participate in the program. Forty-six students responded favorably and attended the course, eight (17.4%) being male and 38 (82.3%) female. The mean age is 20.74 years (SD = 2.20 years), the minimum age being 19 years and the maximum age 31 years. The coefficient of variation for age is 11%, which reveals a homogenous group in terms of age. Regarding the school years of course attendance, 60.90% attend the 2 nd year of CLE, 19.60% the 3 rd year, and 19.60% the 4 th year. As a tool for the pre-and post-intervention, we used the Questionnaire for Assessment of Mental Health Literacy -QuALiSMental (Loureiro, 2015). This questionnaire is composed of different sections relating to each component of mental health literacy. All questions related to MHL are preceded by a vignette reporting a depression case, in accordance with the DSM-5 criteria (American Psychiatric Association, 2006), of an 18-year-old young woman called Joana. The text content of the vignette is as follows: Joana is an 18-year-old young woman who has been feeling unusually sad for the last few weeks. She is always tired and has trouble falling asleep or staying asleep. She has lost her appetite and lately has been losing weight. She has trouble concentrating on her studies, and her grades have sunk. Even daily tasks seem very difficult for her, so she has postponed some decisions. Her parents and friends are very worried about her. (Loureiro et al., 2013, p. 2)

Procedures
The questionnaire was provided in both moments in the training room, with the supervision of the teacher.

Statistical treatment
In this study, we used the IBM SPSS Statistics V24.0 and G-Power 3.1 software programs, the latter for the calculation of observed power (OP) of the post hoc test. We calculated the appropriate summary statistics and used the McNemar test (with the binomial distribution) to test the hypothesis. Subsequently, we calculated the odds ratio (OR) and the effect size measure, using Cohen's g (1988). For the interpretation of results, we used the reference values presented by Cohen (1988 In the tables presented in the results section, the percentages (before and after) correspond to the marginal totals in row and column for the answers marked as "yes" (first and fifth components) or "useful" (second, third, and fourth components).
In global terms, the accurate identification of the situation described in the vignette increased with the attendance of the program, going from 47.80% to 80.40% (p < 0.01; g = 0.30; OP = 0.94). The effect size (g) measures for the statistically significant results reveal a great effect (g ≥ 0.25), with OP estimates of the appropriate tests. As regards the 2 nd component (knowledge about the professionals and available treatments), we observed (Table 1) a statistically significant increase in the perception of the usefulness of the "teacher" (p < 0.05; g = 0.25; OP = 0.74), the "psychiatrist" (p < 0.001; g = 0.43; OP = 0.99), and the "counselling phone service" (p = 0.001; g = 0.39; OP = 0.99). In terms of products and medication, there is still a statistically significant decrease in products not subject to medical prescription such as "teas" (p < 0.01; g = -0.41; OP = 0.93) and a statistically significant increase in "antidepressants" (p < 0.001; g = 0.46; OP = 0.99) and "sleeping pills" (p < 0.05; g = 0.28; OP = 0.81).
In the 3 rd component (knowledge about the effectiveness of self-help strategies; Table 1), we observed an increase in the perception of usefulness of strategies: "getting up early each morning and getting out in the sunlight" (p < 0.001; g = 0.43; OP = 0.99); "checking a website containing information" (p < 0.001; g = 0.50; OP = 1.00); "reading a self-help book about the problem" (p < 0.01; g = 0.31; OP = 0.83); and "joining a support group for people" (p < 0.05; g = 0.29; OP = 0.65). As regards the 4 th component, we observed statistically significant changes, particularly in "ask if he/she is feeling suicidal " (p < 0.001; g = 0.46; OP = 0.99) and "encourage him/her to become more physically active" (p < 0.001; g = 0.50; OP = 1.00).
In the 5 th component (knowledge of how you can prevent mental disorders), we observed (Table 1) a statistically significant increase in the strategies "keeping physically active" (p < 0.05; g = 0.50; OP = 1.00); in "never drinking alcohol" (p < 0.05; g = 0.38; OP = 0.74), and "having a religious and spiritual belief " (p < 0.001; g = 0.41; OP = 0.93). In the item relating to the prevention of mental health problems, through "avoiding situations that might be stressful", we observed a statistically significant decrease (p < 0.01; g = 0.40; OP = 0.91). Note. a Percentages of total marginal in row and column (before and after); * p < 0.05; p < 0.01; p < 0.001 (statistical significance obtained by McNemar test -binomial distribution); † Observed power in the test; a g value = 0.5 corresponds to an OR of infinity and g value = -0.5, corresponds to an OR of zero; OR = Odds Ratio; OP = observed power.
The participants were also questioned, before and after the completion of the PSSM program, about the intention of seeking help (if they were to experience a similar situation), and to what extent they felt confident to provide help to Joana. As shown by Table 2, in the case of intention to seek help, 71.70% initially reported the intention to request/seek help, a value which increased to 91.30% at the end of the program. This increase was statistically significant (p < 0.01; g = 0.50; OP = 1.00). As regards the confidence to provide help, the response format was on a Likert-type scale of 1 (not confident) to 5 points (extremely confident). Evidently, the results of the application of the t-test for paired groups showed statistically significant differences (t (45) = -7.020; p < 0.01; d = -2.09; OP = 0.99), with an improvement in confidence to provide help from the first moment (3.09) to the second moment (4.09). Note. a Effect size measure -Cohen's d; OR = odds ratio; OP = observed power; M = mean; SD = standard deviation.

Discussion
The results of this study justify primarily a reflection about its limitations and weaknesses, particularly those related to the research design (pre-experimental with a single group). The non-existence of a control group and also of follow-up can affect the validity of the study. Another limitation relates to the characteristics of the sample, namely not having an equitable distribution between the 3 school years, since they are mostly second-year students (60.90%), which indicates that there are in the group very different experiences in terms of learning and contact with the field of mental health/ illness. These limitations imply the need for cautious interpretations in the analysis of the program's results. The studies conducted with nursing students and with homogeneous samples in terms of academic year and teaching experiences (Sousa, 2015) show very similar results to those obtained in this study.
If we compare only the pre-test resultswith those obtained in another type of samples, for instance, adolescents and youngsters (Loureiro et al., 2013), the results show a higher mental health literacy in our sample. For example, at the moment before the completion of the program, 40.80% of the students correctly identified the health problem described as depression. In the sample of adolescents and youngsters of middle school and Evaluation of the Mental Health First Aid program in undergraduate nursing students secondary school, 27.20% accurately identify the situation as depression (Loureiro et al., 2013). If we regard the recognition of the problem as a precondition that can encourage the seek for help in mental health (Jorm, 2014), this difference in nursing students suggests, thus, positive changes in the way of coping with mental health problems. The ability of diagnosis is not in question here. That is not our intention nor what is evaluated by the instrument and promoted by the program, but the use of labels indicating the problematization of the problem and the proper recognition of signs and appreciation of symptoms. If we compare the results of the five MHL components with those of other studies, it appears that, generally, 50% of the 50 analyzed items showed statistically significant changes, with emphasis on the increase of recognition and appreciation of mental health problems, skills and confidence to provide first aid, and intention to seek help in mental health. In all cases, the effect size measures reveal high effects, which are consistent with results obtained in other studies which implemented the program in samples of nursing students (Sousa, 2015;Bond et al., 2015;Burns et al., 2017).
In the items with no differences, those differences were not expected in the majority of cases, since students already manifest an adjusted MHL level at the moment before the completion of the PSSM program. For instance, in the knowledge and skills to provide first aid and first help (4 th component), at the initial moment, all students consider useful to "listen to his/her problems in an understanding way." In the inappropriate (harmful) strategies, only a minority considers useful, for example, to "tell him/her firmly to move forward", a value which becomes residual after the intervention. Still regarding this component, the item "ask if he/she is feeling suicidal" is especially relevant. Many people think and are reluctant to talk about suicidal thoughts because they think it is a way to suggest or encourage this type of ideas in people who are suffering when evidence reveals the contrary. When questioned, the person can express his/her feelings, possibly leading to the seeking of professional help (Jorm, 2014;Loureiro, 2014). The results for this component, and specifically the mentioned action strategy are consistent with those of other studies (Sousa, 2015;Bond et al., 2015;Burns et al., 2017), which indicates the effectiveness of the program in this field.
Comparing the results of this study with the evidence produced in the bottom line of the program's effectiveness in other contexts (Hadlaczky et al., 2014;Morgan et al., 2018), we can observe that they do not deviate from those reported in these studies, even with differentiated samples. All studies unanimously claim the program is appropriate and effective in improving MHL.

Conclusion
The results of this study confirm that the MHFA program should be used as an appropriate and effective intervention leading to the increase of mental health literacy in nursing students.
The changes observed in the MHL components are satisfactory and promising, which allows concluding that the program increases the knowledge about mental health problems and depression, as well as the intention to seek help and the confidence to provide mental health first aid and first help.
Further studies about its effectiveness should give priority to experimental designs, with recourse to control groups and experimental groups, randomization of the subjects between groups, and follow-up assessment.
The program is suitable for other contexts/fields of higher education, and not only to health-related courses, even though these groups show evidence of an increased risk of developing mental health problems.