The ophthalmic clinical evaluation exercise: reliability determination

Document Type

Article

Abstract

PURPOSE: Reliable and valid tools must be developed to assess the core residency competencies identified by the Accreditation Council for Graduate Medical Education. The Ophthalmic Clinical Evaluation Exercise (OCEX) is a tool designed to assess the ophthalmology resident's competence in patient care. The OCEX has been shown to have face and content validity. This study will determine the degree to which the OCEX is reliable and has construct validity. PARTICIPANTS: Ninety-four academic ophthalmology teaching faculty from ophthalmology residency programs across the country. METHODS: Participants reviewed a video compact disc of the same resident and new patient encounter and then completed the OCEX. A scoring rubric was provided. RESULTS: Results indicate that the OCEX is a reliable tool for faculty to use to assess residency competency. The coefficient alpha statistic (a measure of reliability/internal consistency) for the OCEX as a whole was 0.81. The alpha statistics for 3 of 4 subscales that comprise the OCEX (i.e., interviewing skills = 0.65, interpersonal skills/professionalism = 0.73, case presentation = 0.70) were lower than the OCEX as a whole, but were acceptable for new scales. However, the alpha for the examination subscale (i.e., 0.27) was extremely low. Interrater reliability assessment shows that of 33 individual OCEX items, 31 (94%) had at least 85% of the raters rating the student in 1 of 2 consecutive rating categories. CONCLUSIONS: The OCEX shows both reliability and validity and, therefore, meets the Accreditation Council for Graduate Medical Education criteria for an acceptable assessment tool.

Medical Subject Headings

Accreditation (standards); Clinical Competence (standards); Education, Medical, Graduate (standards); Educational Measurement (standards); Humans; Internship and Residency (standards); Ophthalmology (education); Program Evaluation; Quality of Health Care; Reproducibility of Results; Specialty Boards (standards); United States

Publication Date

10-1-2005

Publication Title

Ophthalmology

E-ISSN

1549-4713

Volume

112

Issue

10

First Page

1649

Last Page

54

PubMed ID

16111754

Digital Object Identifier (DOI)

10.1016/j.ophtha.2005.06.006

Link to Full Text

Share

COinS