Translation and cross-cultural adaptation of evidence-based practice instruments for Portuguese nursing students

Background: The implementation of evidence-based practice (EBP) in clinical contexts is recommended due to its positive impact on health, but it remains under the desirable. The training of undergraduate nursing students in the use of EBP is crucial, and, for that, there must be valid and reliable measures of EBP learning. Objective: To translate and to cross-cultural adapt into European Portuguese of the EBP Beliefs Scale (EBPB), EBP Implementation Scale for Students (EBPI-S), and Organizational Culture & Readiness for School-wide Integration of Evidence-based Practice Survey for Students (OCRSIEP-ES). Methodology: Translation and cross-cultural adaptation according to international recommendations. Preliminary validation in Portuguese undergraduate nursing students from nine institutions. Results: In the pre-final versions of the instruments, the participants suggested including the optional answer “I do not have sufficient knowledge to be able to answer” and increasing the recall period in the EBPI-S instrument. Phase 2 included 167 participants. The α for EBPB, EBPI-S, and OCRSIEP-ES was 0.854, 0.943, and 0.970, respectively. Conclusion: Preliminary results showed good internal consistency. Further validation studies with robust samples are required to test the reliability and validity of the instruments.


Introduction
The concept of Evidence-Based Medicine arose in the area of medicine but was quickly embraced by other healthcare professions. As a consequence, terms such as evidence-based practice (EBP), evidence-based healthcare (EBHC), and evidence-based nursing (EBN) emerged (Dawes et al., 2005). There are many definitions of EBP, but the following three elements are always present in most of them: use of best available evidence, use of clinical/ professional expertise, and patient involvement (International Council of Nurses [ICN], 2012; Pearson, Jordan, & Munn, 2012). In the last decades, the use of EBP in clinical practice has been a focus of particular attention due to various concomitant factors, like the acknowledgement of the positive impact of EBP on healthcare, the ever-growing production of new primary research, the well-known delay in incorporating new evidence into clinical practice, the healthcare quality and safety movement, and the pressure of health service users with quick access to health information (Dawes et al., 2005;Melnyk, Gallagher-Ford, Long, & Fineout-Overholt, 2014). Consequently, several organizations have encouraged the implementation of EBP in clinical contexts (World Health Organization [WHO], 2015;ICN, 2012). Despite these recommendations, there is still a less than desired translation of evidence into clinical practice by nurses (Duncombe, 2018;Melnyk, Fineout-Overholt, Gallagher-Ford, & Kaplan, 2012). In an earlier phase, many studies with nurses working in clinical settings were conducted to support the integration of EBP in clinical practice. However, following the recommendations of the Sicily Statement on Evidence-Based Practice for EBP teaching and education (Dawes et al., 2005), the integration of EBP teaching in nursing curricula has gained a spotlight in recent years. Undergraduate nursing students will be the future health professionals and, as a result, play a crucial role in influencing the use of EBP in healthcare contexts in the future. Therefore, it is essential to understand the undergraduate nursing students' beliefs regarding EBP, the level of their EBP implementation skills, and their perception of the state of readiness for school-wide EBP integration to develop teaching strategies for EBP use promotion. However, there are no available instruments in Portugal for this purpose. Therefore, to tackle this shortcoming, the objective of this study is the translation and cross-cultural adaptation into European Portuguese of the instruments EBP Beliefs Scale (EBPB), EBP Implementation Scale for Students (EBPI-S), and Organizational Culture & Readiness for School-wide Integration of Evidence-based Practice Survey for Students (OCRSIEP-ES). Also, this study aims to provide preliminary validation data of the European Portuguese versions.

Background
Nowadays, the use of EBP in clinical practice remains less than desirable, despite the strong recommendations for it. Many studies have reported both barriers and facilitators to the integration of EBP in clinical practice. The identified barriers include lack of time; organizational culture and the philosophy of "that is how we have always done it here"; lack of EBP knowledge, as in lack of search skills and lack of confidence in assessing research quality; difficulties in statistical interpretation; lack of resources (e.g., no access to evidence); manager/leader and co-worker resistance to change practices; and heavy workload (Melnyk et al., 2012;Pereira, Cardoso, & Martins, 2012;Solomons & Spross, 2011). As to the facilitators, the following were identified: education (e.g., training in research methods and EBP); organizational support/awareness; collaboration between EBP mentors and clinical staff to implement best practices; time availability; resource availability, like access to evidence (Duncombe, 2018;Melnyk et al., 2012). Education appears as a facilitator of EBP integration into clinical practice. Melnyk et al. (2004) reported that "knowledge and beliefs about EBP are related to the extent that nurses engage in EBP" (p. 190). Moreover, in 2005, the recommendations of the Sicily Statement on Evidence-Based Practice for EBP teaching and education highlighted that "all health care professionals need to understand the principles of EBP, recognise EBP in action, implement evidence-based policies, and have a critical attitude to their own practice and to evidence." (Dawes DANIELA FILIPA BATISTA CARDOSO et al. et al., 2005, p. 4). Because undergraduate nursing students will be healthcare professionals in the future, their time spent in nursing school is an absolutely unique opportunity to instill in them the EBP culture. This is why training undergraduate nursing students in the use of EBP is imperative, and, for that, there must be valid and reliable measures of EBP learning. The "Sicily Statement on Classification and Development of Evidence-Based Practice Learning Assessment Tools" (Tilson et al., 2011) was designed to guide the development of EBP assessment tools. The following assessment categories were suggested: Benefit to patients; Behavior; Skills; Knowledge; Self-efficacy; Attitudes; and Reaction to the educational experience. The Benefit to patients category refers to the assessment of health outcomes of patients and communities. The Behavior category could contribute significantly to the identification of students' learning needs regarding the use of EBP. The Skills category concerns knowledge applicability when performing an EBP-related task. The Knowledge category related to the preservation of EBP-related facts and concepts by learners. The Self-efficacy category includes the perceptions of individuals regarding their ability to perform a specific activity. The Attitudes category concerns the values acknowledged by the student of the relevance and usefulness of EBP to inform clinical decision-making. Lastly, the Reaction to the educational experience category related to the learners' perceptions of the learning experience, including aspects like the relevance of organization for an effective education intervention (Tilson et al., 2011). The last of the four general recommendations for developers of new EBP learning assessment tools presented in the aforementioned statement is "Develop, validate, and use a standardized method for translation of tools into new languages." (Tilson et al., 2011, p. 8). In this sense, and considering that there are no tools available in Portugal to assess the undergraduate nursing students' beliefs regarding EBP, the level of their EBP implementation skills, and their perception of the state of readiness for school-wide EBP integration, this study aims at the translation and cross-cultural adaptation of the EBPB, EBPI-S, and OCRSIEP-ES tools into European Portuguese.

Research questions
Do the European Portuguese versions of the EBPB, EBPI-S, and OCRSIEP-ES reflect the original versions of the tools adequately? Is the internal consistency of the European Portuguese versions of the EBPB, EBPI-S, and OCR-SIEP-ES acceptable?

Methodology
This study was conducted during 2017-2018 and comprised two phases: Phase 1 -translation and cross-cultural adaptation of the three instruments into European Portuguese; Phase 2 -preliminary validation of these versions in Portuguese undergraduate nursing students.

Instruments
The EBPB, EBPI-S, and OCRSIEP-ES tools were developed by Fineout-Overholt and Melnyk (Fineout-Overholt, 2018). The EBPB tool assesses undergraduate nursing students' EBP-related beliefs and their confidence in their EBP implementation ability. It is a 16-item instrument with a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree), whose score ranges from 16 to 80 (the higher the score, the stronger the beliefs). Two of the 16 items are reverse-scored items (Item 11 -"I believe that EBP takes too much time." and Item 13 -"I believe EBP is difficult."). The EBPI-S is an 18-item self-report tool that assesses the EBP implementation skills of undergraduate nursing students considering their engagement in the desired EBP behaviors during the last eight weeks. The EBPI-S total score ranges from 0 to 72. Each item is scored with a 5-point scale (0 = 0 times; 1 = 1-3 times; 2 = 4-5 times; 3 = 6-8 times; 4 = more than 8 times). The OCRSIEP-ES is a 25-item self-report tool that measures the students' perception of the state of readiness for school-wide integration of EBP and its influencing factors. Each item is scored with a 5-point scale (1 = none at all to 5 = very much), and the total score ranges from 25 to 125.

Phase 1 -Translation and Cross-Cultural Adaptation
The translation and cross-cultural adaptation of the EBPB, EBPI-S, and OCRSIEP-ES into Eu-Translation and cross-cultural adaptation of evidence-based practice instruments for Portuguese nursing students ropean Portuguese were performed according to the guidelines provided by Beaton, Bombardier, Guillemin, and Ferraz (2000) for the cross-cultural adaptation of self-report measures. These guidelines recommended the following five stages. Stage 1 -Initial translation: independent translation by two bilingual translators (one familiar with the concepts, and the other a naive translator). Stage 2 -Synthesis of the translations: preparation of a standard translation. Stage 3 -Back-translation: two translators performed, independently, the back-translations. Both translators were not aware of the concepts being measured.
Stage 4 -Expert committee: 7 experts (health professionals, individuals experienced in validation studies, a language professional, and translators) analyzed the tools' versions and developed the pre-final versions. One of the original authors of the instruments (Dr. Ellen Fineout-Overholt) was contacted to clarify ambiguous items and the meaning of some terms or expressions. Stage 5 -Testing of the pre-final versions: The pre-final versions were tested in a sample of undergraduate nursing students. Each participant completed the instruments and a brief questionnaire (Figure 1) regarding the tools´ comprehensibility.
In your opinion: Do you consider that the items' statements are written clearly?
Yes No If not, which are not clear and why?
If you did not rate one or more items, please list them and identify the reason(s) using the following statements: -I did not rate item(s) ________________ because I have no sufficient knowledge to be able to answer.
-I did not rate item(s) ________________ because the statement is not clear.
-I did not rate item(s) ________________because__________________ If you wish, please leave any additional comments:

Phase II -Preliminary validation
Undergraduate nursing students from nine Portuguese nursing schools participated in this phase. The three leading Portuguese nursing education institutions (not integrated into a polytechnic institute or university) were selected by convenience. The remaining six institutions were randomly selected (one institution from a polytechnic institute and one from a university in each region of Continental Portugal -north, center, and south). Nine socio-demographic questions and the three Advancing Research & Clinical practice through close Collaboration in Education (ARCC-E) questionnaires (total of 59 items) were included in the online survey.

Statistical analysis
All statistical analysis was carried out in the IBM SPSS Statistics program (version 24.0; SPSS Inc., Chicago, IL, USA). Descriptive analysis was used for sample characterization purposes, such as mean, standard deviation, minimum, maximum, and percentages. The internal consistency was assessed using the Cronbach's alpha coefficient.

Ethical consideration
This study was approved by the Ethical Committee of the Faculty of Medicine of the University of Coimbra (no. CE-037/2017). The original authors of the instruments have consented their use. The institutions provided written approval. All participants provided informed consent, and the data were subject to confidential treatment.

Phase 1 -Translation and Cross-Cultural Adaptation
The first three stages of the phase of transla- tion and cross-cultural adaptation carried on smoothly. At stage 4, the expert panel discussed and suggested some modifications to clarify and adapt the instruments into the Portuguese context. Generally, the expert panel agreed upon the use of the term "utentes" for the translation of "patients" because, in Portugal, that term is more suitable when referring to a user of health services regardless of whether one is ill or not. The expression "evidence-based guidelines" was translated as "diretrizes/orientações (guidelines) baseadas em evidência." However the expert panel decided to keep the term "guidelines" between brackets, since the meaning of this loanword is widely known in Portugal. Similarly, the expert panel decided to keep the English phrase "critically appraising" between brackets following its translation ("avaliação da qualidade metodológica") for a better understanding of the phrase "critically appraising evidence." The phrase "a time-efficient way" was challenging to translate. The expert panel decided to translate it as "adequadamente e em tempo útil" to remain faithful to the original meaning. Specifically, the EBPI-S items 12 and 13 were adapted to accept other systematic review and guideline databases besides the Cochrane database of systematic reviews and the National Guidelines Clearinghouse. Item 12 was adapted from "Accessed the Cochrane database of systematic reviews…" to "Acedi a base(s) de dados de revisões sistemáticas (por exemplo, Cochrane database of systematic reviews)" and the item 13 from "Accessed the National Guidelines Clearinghouse…" to "Acedi a base(s) de dados de diretrizes/orientações (guidelines; por exemplo, National Guidelines Clearinghouse)". As regards the OCRSIEP-ES, the following note was added to explain the meaning of "mentor": "Mentores de PBE: pessoa confiável com conhecimentos e treino avançado em PBE que orienta, promove a autoconfiança e infunde valores no aprendiz." In addition, some terms or expressions were paraphrased for the Portuguese nursing education context, such as "community partners" (instituições parceiras onde decorrem os ensinos clínicos/prática clínica), "didactic course faculty" (corpo docente das unidades curriculares teóricas, teórico-práticas, práticas), and "clinical course faculty" (corpo docente dos ensinos clínicos/ prática clínica). Following all these changes, the expert committee agreed upon the pre-final versions of the Portuguese translation. Thirty-seven Portuguese undergraduate nursing students, five males and 32 females aged between 18 and 27 years, participated in stage 5. Overall, the students understood the meaning of the items, but many of them (mainly from first and second years) reported insufficient knowledge to choose an optional answer. They also reported that the time of application of the EBPI-S could influence the answer, whether they are at school or in clinical practice. The comments provided by the students were analyzed, and in response to their concerns the optional answer "I do not have sufficient knowledge to be able to answer" (Não tenho conhecimento suficiente que me permita responder) was added and the EBPI-E recall period was changed from 8 weeks to one year.

Phase 2 -Preliminary validation
A total of 167 undergraduate nursing students with a mean age of 22.13 years (SD = 4.20; range: 18 -45) completed the online questionnaire. A large majority of this sample was female (n = 140, 83.8%) and has completed the 12 th grade (n = 159, 95.2%). More than half of the sample participated in EBP training programs (n = 88, 52.7%). Of these 88 participants, 76 stated that the training was integrated into the curricula, five stated that it was an extracurricular activity, and seven that it was both. The majority of the participants came from the main Portuguese nursing education institutions (n = 118, 70.66%; Table 1).
Translation and cross-cultural adaptation of evidence-based practice instruments for Portuguese nursing students

Preliminary validation of the EBPB
Regarding the EBPB internal consistency analysis, 63 participants were excluded for responding, "I do not have sufficient knowledge to be able to answer" in one or more items. Consequently, 104 participants with a mean age of 22.59 years (SD = 4.14; range: 18 -43) remained. A vast majority of this sample was female (n = 85, 81.7%), and more than half of it had completed the 12 th grade (n = 97, 93.3%), 5 held a bachelor's degree (4.8%), and 2 held a master's degree (1.9%). The majority of this sample participated in EBP training programs (n = 68, 65.4%) and came from the main Portuguese nursing schools (n = 77, 74.0%). Of the 104 individuals, 10 were first-year undergraduate students, 12 were second-year students, 36 were third-year students, and 46 were fourth-year students.
The 63 excluded individuals had a mean age of 21.37 years (SD = 4.22; range: 18 -45). A vast majority of these students were female (n = 55, 87.3%), had completed the 12 th grade (n = 62, 98.4%), and came from the main Portuguese nursing schools (n = 41, 65.1%). However, the majority of them did not participate in EBP training programs (n = 45, 71.4%). Of the 63 students, 29 were first-year undergraduate students, 8 were second-year students, 18 were third-year students, and 8 were fourth-year students. The EBPB item means ranged between 2.70 (item 13) and 4.60 (item 1). The EBPB presented a good internal consistency (α = 0.854), and the corrected item-total correlations ranged between 0.181 and 0.733, meaning a poor to good correlation between the items and total score (Table 2). Preliminary validation of the EBPI-S Seventy-three participants were excluded from the EBPI-S internal consistency analysis because they answered, "I do not have sufficient knowledge to be able to answer" in one or more items. Therefore, 94 participants with a mean age of 22.41 years (SD = 3.83; range: 18 -43) remained. A large majority of this sample was female (n = 77, 81.9%), and more than half of it had completed the 12 th grade (n = 91, 96.8%) and participated in EBP training programs (n = 62, 66.0%). The majority came from the main Portuguese nursing schools (n = 71, 75.6%). Eight students frequented the 1 st year of the bachelor's degree, nine the 2 nd year, 34 the 3 rd year, and 43 the 4 th year. The 73 excluded individuals had a mean age of 21.75 years (SD = 4.63; range: 18 -45). A vast majority of this sample was female (n = 63, 86.3%) and had completed the 12 th grade (n = 68, 93.2%). However, they did not participate in EBP training programs (n = 49, 67.1%). Forty-seven individuals (64.3%) came from the main Portuguese nursing schools; eight (11.0%) came from nursing schools integrated into polytechnic institutes, and 18 (24.6%) came from nursing schools integrated into universities. Many of these participants were first-year undergraduate students (n = 31). The remaining participants frequented the second year (n = 11), the third year (n = 20), and the fourth year (n = 11). The item means of the EBPI-S ranged between 1.03 (item 10) and 2.69 (item 1). The EBPI-S presented an excellent internal consistency (α = 0.943), and the corrected item-total correlations ranged between 0.308 and 0.808, meaning an acceptable to good correlation between the items and total score (Table 3).
Translation and cross-cultural adaptation of evidence-based practice instruments for Portuguese nursing students

Preliminary validation of the OCRSIEP-ES
Regarding the internal consistency analysis of the OCRSIEP-ES, 121 participants were excluded for responding, "I do not have sufficient knowledge to be able to answer" in one or more items.

Discussion
The authors believe that the EBPB, EBPI-S, and OCRSIEP-ES are the first instruments translated into European Portuguese to assess undergraduate nursing students' beliefs regarding EBP, their level of EBP implementation skills, and the state of readiness for school-wide EBP integration. Generally, the translation and cross-cultural adaptation of the three instruments was a smooth process and encountered no problems for the majority of the items. Similar to some data reported by Fineout-Overholt (2018), the European Portuguese versions of the three instruments showed good internal consistency with Cronbach's alphas ≥ 0.85. However, there is no available additional information regarding studies that used the original instruments that allows performing more detailed comparisons. During stage 5 of the phase of translation and cross-cultural adaptation, two recommendations of the participants were accepted by the authors and should be discussed. One related to the inclusion of the optional answer, "I do not have sufficient knowledge to be able to answer," and the other suggested changing the recall period of the EBPI-S from 8 weeks to 1 year.
Translation and cross-cultural adaptation of evidence-based practice instruments for Portuguese nursing students The authors decided to include the optional answer "I do not have sufficient knowledge to be able to answer" in all scales because the first recommendation was made during the pre-test, and they were aware of the lack of knowledge of potential participants to answer some items. Furthermore, it was evident that, if this optional answer were not provided to the participants, many of them might be forced to guess the answer, thus leading to data contamination. However, it should be noted that the optional answer "I do not have sufficient knowledge to be able to answer" in each scale was not scored, and if participants chose this answer in at least one item, they were removed from the analysis. It means that this optional answer contributed to ensuring that only the participants who perceived they had the necessary knowledge to answer the items were included in the analysis. This benefits the descriptive analysis in the specific context of Portugal.
As regards the suggestion of stage 5 participants to extend the recall period of the EBPI-S, the authors decided to change this recall period to 1 year. The participants claimed that the answer could be influenced by the moment when the EBPI-S is applied, depending on whether they are at school or in clinical practice. However, some authors showed that data accuracy decreases as the recall period increases (Clarke, Fiebig, & Gerdtham, 2008;Stull, Leidy, Parasuraman, & Chassany, 2009) since long recall periods lead to participants guessing the answer (Brown, 2002;Blair & Burton, 1987). Therefore, a one-year recall period could be too long for recall reliability, and, as a result, participants may answer the instrument taking into account only the social acceptability. Indeed, this change of the recall period challenges the validity of the EBPI-S because it was not designed for such a long recall. This study presents some limitations. First, it used a small sample size. According to Streiner and Norman (2008), the sample size should be 300 participants for a Cronbach's alpha of 0.70 and a confidence interval of ±0.10. Also, at least 250 participants are necessary (the scale with the most items has 25) to carry out the exploratory and confirmatory factor analysis because the sample size should have a ratio of 10 participants per item (Tinsley & Tinsley cited by DeVellis, 2016). Moreover, extending the recall period of the EBPI-S may have affected the validity of the tool.

Conclusion
To the best of our knowledge, the European Portuguese versions of the EBPB, EBPI-S, and OCRSIEP-ES are the first instruments translated into European Portuguese to assess undergraduate nursing students' EBP beliefs, their level of EBP implementation skills, and their perception of the readiness for school-wide integration of EBP. The translation and cross-cultural adaptation used a rigorous methodology that ensured the structural, linguistic, and cultural equivalences between the original versions and the European Portuguese versions of the three scales. These European Portuguese versions showed a good internal consistency and low to excellent correlations between the items and total score. The translation and cross-cultural adaptation of the EBPB, EBPI-S, and OCRSIEP-ES are the first contributions to having valid and reliable measures of EBP learning for Portuguese undergraduate nursing students. However, more research studies for validation of the European Portuguese versions of those tools should be conducted with larger sample sizes to test their measurement properties.