|
Abstract
The Objective Structured Clinical Examination (OSCE) has been a common tool of assessment in both undergraduate and postgraduate medicine, and has been reported to have both higher reliability and validity over the oral exam. In addition, another advantage is that it reduces luck by standardizing both examiners and patients. This article describes our experience in organizing and conducting an OSCE for Oman Medical Specialty Board residents in Psychiatry.
Keywords: OSCE; Psychiatry; Postgraduate training.
Introduction
The Objective Structured Clinical Examination (OSCE) has been a popular assessment tool in undergraduate as well as postgraduate medicine, and has been reported to have both higher reliability and validity over the oral exam,1 also an additional advantage is that it reduces luck by standardizing both examiners and patients. The OSCE was introduced in 1975 by Harden and Gleeson,2 and since then it became widely used for assessment in both undergraduate and postgraduate examinations in medicine. OSCE is best defined as "an approach to the assessment of clinical competence in which the components of competence are assessed in a planned or structured way". It allows examiners to assess the wide range of skills and behaviors of the candidates under varying degrees of difficulty and complexity by observing them interact with a standardized real or simulated patient. It is believed to be superior to oral examination as it allows multiple competencies to be assessed in one exam, and eliminate subjectivity that characterize other methods of assessment.3 The use of standardized patients allows for the level of difficulties to be adjusted to the candidates training level.
The OMSB Psychiatry Program introduced OSCE this year for the first time as part of end of year examinations for all residents; however the Department of Behavioral Medicine - College of Medicine and Health Sciences at Sultan Qaboos University started OSCE for undergraduate medical students in 2002.4 Since then we have conducted around 80 OSCE exams. This experience was of great help to us in setting the exam for OMSB residents despite the differences in the level of experiences and competencies tested.
Station design
The OSCE stations were prepared by 2 consultant psychiatrists using a standardized blue print provided by the OMSB. We agreed to use 5 OSCE stations and 3 short cases, with each station allocated 7 minutes. Cases were selected from different clinical psychiatric situations covering different levels of complexity for each of the residents’ group (residents years 1 and 2, and residents years 3-5). Each of the 5 stations had a simulated patient who has been briefed about the role they are playing by one of the organizers. A scoring sheet (checklist) was prepared for each station containing the expected key answers. The candidate was allowed 1 minute to read the case scenario and the task in question. An examiner observed the candidate interaction with the simulated patient and marked the scoring sheet accordingly. The sheet also contained a section for the examiner to comment about the candidate’s attitude and communication skills. Examiners were instructed not to talk or to give hints to the candidates during the exam.
Two sets of questions were prepared to assess competency levels for both first and second year residents (R1&R2) arranged in two groups (A&B), and third, fourth and fifth year residents (R3 - R5) arranged in two groups also (C&D). A break was allocated between the two groups to prepare the simulated patients for the new role. The stations for R1&R2 focused on assessing history and physical examination, while that for R3-R5 focused on management issues.
The short case
A case vignette was introduced to the resident followed by questions from each of the 2 examiners. Residents’ answers were marked in the scoring sheet, examiners were encouraged to discuss the score among themselves during the break, in case of discrepancy in the marking. Both scores are added and an average is used in the final marking. (Tables 1 and 2)
Table 1: A sample of an OSCE and short case stations.
Table 2: Residents feedback about the OSCE end of year exam.
Standardized patients
We often used real patients when conducting OSCE for undergraduate students; however this OSCE was conducted on a weekend to ensure maximum availability of senior examiners, so it was difficult to invite real patients. Instead nurses and orderlies from the department were requested to act as simulated patients. They were briefed about the cases early in the day.
Venue and introduction
The exam was conducted in the Psychiatry Outpatient Clinic bay area at Sultan Qaboos University Hospital where rooms are set in a U-shaped layout with an open area in the middle. This allowed residents to move smoothly between rooms while an organizer sat in the middle area to coordinate the running of the exam. A staff member was asked to set the alarm clock every 7 minutes announcing time to move to the following station. The big seminar room was used to accommodate the subsequent groups of residents to be examined when the first group finished the exam.
Candidates
Twenty-six residents in Psychiatry program from all training years attended the exam (5 males and 21 females). All candidates had previous experience with OSCE during their psychiatry rotation as 6th year medical students. All residents were encouraged to attend a workshop about OSCE as part of their psychiatric training program. This included general information about the structure of the OSCE and the competencies tested. Residents were exposed to a variety of stations covering several areas in General Psychiatry and subspecialties that are appropriate to their training level.
The OSCE circuit
The 26 residents who attended the exam were divided into 4 groups (A, B, C & D). Group A arrived at 8:30, received a short briefing about the exam, and received the questions’ handbook and stickers with their names and individual OMSB numbers which they have to hand to the examiner in the OSCE and short case stations. They were instructed to read the question before entering the room. An alarm clock was set to ring every 7 minutes announcing the time to move from one station to another.
The candidates were offered refreshments at the end of the exam and were given feedback forms to fill in. They were asked to wait in a separate area while the remaining groups entered the exam area; this is done to avoid any leakage of the exam questions. A 10 minute break was set to allow examiners and simulated patients to have a short rest. Examiners' feedback was collected at the end of the day.
Marking the OSCE
The initial marking takes place during the OSCE station by the observing examiner. At the end of the exam, all scores are added by an allocated trainer.
Challenges and Limitations
The examination date was set during the summer (July) when most trainers were on leave, however we managed to invite 13 examiners, two senior examiners sat in every short case station and 5 examiners in the OSCE stations, one for every station. Ideally we would have preferred to have real patients participating in the OSCE, however this was made difficult by the fact that the exam was conducted on a weekend. The use of healthcare worker as simulated patients did not have a major impact on the test as they were briefed about the symptoms and other information’ they were expected to provide to the candidates when asked. The simulated patient were not expected to act our behavioral symptoms
Simulated patients received incentives for their participations, which is widely practiced behavior in other centers that use OSCE in their examinations.
Conclusion
We found the experience of conducting the OSCE in the Psychiatry end of year exam 2010-2011 for the residents was exciting for both the trainers and the trainees, which proved to be very useful and applicable. OSCE is a valid and reliable examination tool that can assess several skills and competencies in one exam. Selecting case scenarios from day to day clinical encounter makes it more relevant and valid in assessing the competency of an individual candidate.
It is important to have an OSCE working group so that members share different tasks and cover for leave or sickness. Exposing the residents to regular mock exams as part of their academic program reduces their anxieties during the real exam and ensures an ongoing skills development. Finally, we consider this trial very successful and with further improvements, a valuable tool for future assessments.
Acknowledgements
The authors reported no conflict of interest and no funding was received for this work. |