"You Need to See It": Attitudes and Preferences of Health Sciences Librarians Regarding Assessment of Evidence-Based Medicine via an Objective Structured Clinical Examination

Objectives: While the ability of trainees to demonstrate competence in Evidence Based Medicine (EBM) is an expectation, concerns have surfaced regarding medical schools’ ability to provide comprehensive assessment. Previous efforts to improve this led to the development of a tool and process to assess EBM behaviors via observation in an Objective Structured Clinical Examination (OSCE). The next step in this process is evaluating its feasibility for wider adoption across institutions. The purpose of this study is to understand and document the experiences and preferences of health sciences librarians regarding use of OSCEs in assessing medical students’ EBM behaviors.

Methods: We used a grounded theory approach to explore librarians’ perceptions of assessing competence of medical students in practicing evidence-based medicine using an OSCE. Seventeen librarians who had participated in such assessment were invited to participate in focus groups about their experiences. All participants had received training on the background and use of the OSCE assessment form and process, were provided samples to assess in order to calibrate, and then each participated in assessing medical students as a primary or secondary assessor. During the focus groups, the interviewer asked open-ended questions, inviting participants to describe their roles in teaching and assessing EBM at their home institutions, gaps and opportunities they see in EBM assessment, and thoughts on the potential for expanding the use of the OSCE. All interviews were audio recorded and transcribed verbatim for analysis. The authors conducted a round of initial coding to surface main themes, which were used to develop a formal coding structure. A second round of focused coding using this structure was completed, using the Dedoose Desktop App to store and code the transcripts.

Results: Analysis revealed high variability in existing EBM assessment activities and the role of the library across institutions, with many common challenges. Time constraints, limited staffing, and the need for administrative support and faculty partnerships were cited as barriers to successful, scalable EBM instruction and assessment. Benefits of the OSCE in addressing these barriers were explored, including the impact of results on student attitudes and faculty awareness regarding EBM skills. Benefits and challenges of implementation were also addressed, and specific feedback was collected on how to improve the experience and implementation of the OSCE.

Conclusions: Wider adoption of the EBM OSCE across institutions is feasible and would be welcomed by health sciences librarians, but buy-in and support from administration is needed to make this happen. Evidence of student level of ability from existing implementations of the OSCE may be helpful in attaining buy-in and securing more time for libraries within curricula. Cross-institution collaborations to support a standard for assessing EBM behaviors will also improve the reliability of using this structured rubric.