Whole-task assignment for pre-clinical students: assessing EBM skills before clerkships

Objectives: As medical students develop clinical expertise and begin to provide direct patient care, it is important that they are able to apply Evidence Based Medicine (EBM) principles to practice. Whole-task EBM assignments have been associated with improved learning and application of EBM skills, but tend to be applied in the clerkships and beyond. The aim of this study is to determine effectiveness of course-integrated EBM instruction as evidenced by student performance on a capstone whole-task assignment at the end of a six-part EBM course series embedded in the first-year, pre-clinical curriculum.

Methods: At the end of a series of seven EBM-focused sessions embedded in a first-year Practice of Medicine course, students were asked to complete a whole-task assignment. This assignment requires demonstration of EBM skills that had been covered in the course, including: formulating a question based on a patient scenario; documenting their database searches when attempting to acquire evidence; selecting an article to answer their question; appraising the article; and discussing how they would (or would not) apply that evidence to the patient in the scenario. They were also asked to reflect on their process and assess their own skills. Students submitted their completed assignments via the School’s learning management system, where two librarians reviewed them and provided structured individual feedback using a rubric adapted from the Fresno test. The librarians analyzed the complete set of assignment results to identify patterns and provide a holistic snapshot of student performance.

Results: 95 first-year students completed the assignment, out of 111. A preliminary assessment of their overall performance showed 15% of students scoring below expectations, 56% meeting expectations, and 29% exceeding expectations. Mechanical errors in basic question formulation and syntax for database searching were revealed as common challenges, while selection and appraisal of evidence provided insight into students’ strengths and gaps in understanding. 88% of students chose a recent systematic review; 70% selected what the librarians determined as best evidence available on the topic. Analysis is ongoing. Further details regarding performance on distinct EBM steps will be included in the final presentation.

Conclusions: Results from the whole task assignment provided a useful demonstration of students’ abilities at the end of the first-year EBM course. This provided librarians with insights into areas of improvement for future iterations of the course, and enabled us to provide constructive feedback to students to guide their continued growth. This information can also be used to enhance communication with faculty, increasing awareness of students’ level of mastery and informing the continuity of EBM content into Clerkships and beyond. Future studies will connect this whole-task assessment framework to other EBM assessments to track student performance and provide feedback on EBM skills throughout their education.