Advertisement

Using a rapid-cycle approach to evaluate implementation of competency-based medical education in ophthalmology

Published:November 10, 2022DOI:https://doi.org/10.1016/j.jcjo.2022.10.011

      Abstract

      Objective

      As competency-based medical education is being implemented across Canada, there is an increasing need to evaluate the progress to date, including identification of strengths and weaknesses, to inform program development. Ophthalmology is preparing for a national launch in coming years. The purpose of this study was to describe key stakeholders’ lived experiences in the competency-based medical education foundation-of-discipline stage in one ophthalmology department.

      Design

      Using a case-study approach, a qualitative rapid-cycle evaluation was conducted during the 2018–2019 academic year.

      Participants

      Residents, faculty, academic advisors, competence committee members, the program director, the program administrator, and the educational consultant were invited to participate in the program evaluation.

      Methods

      The rapid-cycle evaluation consisted of 2 evaluation cycles, with the first round of interviews and focus groups occurring in October 2018 and the second round in March 2019. Recommendations were implemented in November 2019 and June 2019. All data were analyzed thematically using NVivo.

      Results

      Three main themes emerged across all data sets: developing a shared understanding (e.g., role expectations and changes to assessment), refining assessment processes and tools (e.g., the need for streamlining and clarification), and feedback (e.g., perceived benefits and value of narrative comments).

      Conclusions

      Exploring lived experiences in this study resulted in positive and immediate improvements to the residency program. The recommendations and approach will be useful to other Canadian departments and institutions as they prepare for Competence by Design.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Canadian Journal of Ophthalmology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Carraccio C
        • Englander R
        • Van Melle E
        • et al.
        Advancing competency-based medical education: a charter for clinician-educators.
        Acad Med. 2016; 91: 645-649
        • ten Cate O.
        Competency-based medical education.
        in: Cockerham WS Dingwall RS Quah SR The Wiley-Blackwell encyclopedia of health, illness, behavior and society. Wiley Blackwell, Hoboken, NJ2014: 1329-1335
        • Frank JR
        • Snell LS
        • Cate OT
        • et al.
        Competency-based medical education: theory to practice.
        Med Teach. 2010; 32: 638-645
        • Wentzell D
        • Chung H
        • Hanson C
        • Gooi P.
        Competency-based medical education in ophthalmology residency training: a review.
        Can J Ophthalmol. 2020; 55: 12-19
        • Braund H
        • Dalgarno N
        • McEwen L
        • Egan R
        • Reid MA
        • Baxter S.
        Involving ophthalmology departmental stakeholders in developing workplace-based assessment tools.
        Can J Ophthalmol. 2019; 54: 590-600
        • Caverzagie KJ
        • Nousiainen MT
        • Ferguson PC
        • et al.
        Overarching challenges to the implementation of competency-based medical education.
        Med Teach. 2017; 39: 588-593
        • Dagnone J
        • Chan M
        • Meschiino D
        • et al.
        Living in a world of change: bridging the gap from competency-based medical education theory to practice in Canada.
        Acad Med. 2020; 95: 1643-1646
        • Deepak D
        • Iobst W.
        The importance of faculty development in the transition to competency-based medical education.
        Med Teach. 2010; 32: 683-686
      1. Royal College of Physicians and Surgeons of Canada (RCPSC). What is competence by design? [Internet]. 2022. Available at: https://www.royalcollege.ca/rcsite/cbd/what-is-cbd-e. Accessed 10 November 2022.

      2. Royal College of Physicians and Surgeons of Canada (RCPSC). Competence by design [Internet]. 2020. Available at:http://www.royalcollege.ca/rcsite/cbd/competence-by-design-cbd-e. Accessed 10 November 2022.

        • Dagnone D
        • Stockley D
        • Flynn L
        • et al.
        Delivering on the promise of competency based medical education: an institutional approach.
        Can Med Educ J. 2019; 10: e28-e38
      3. Royal College of Physicians and Surgeons of Canada (RCPSC). Objectives of training in the specialty of ophthalmology: RCPSC [Internet]. 18 Jul 2022. Available at: https://www.royalcollege.ca/rcsite/documents/ibd/ophthalmology-otr-e. Access 10 November 2022.

        • Frank J
        • Snell L
        • Sherbino J
        • et al.
        Draft CanMEDS 2015 milestones guide.
        Royal College of Physicians and Surgeons of Canada, Ottawa2014
        • Holmboe E
        • Sherbino J
        • Englander R
        • Snell L
        • Frank J.
        A call to action: the controversy of and rationale for competency-based medical education.
        Med Teach. 2017; 39: 574-581
        • Gruppen L
        • Frank J
        • Lockyer J
        • et al.
        Toward a research agenda for competency-based medical education.
        Med Teach. 2017; 39: 623-630
        • Hamza D
        • Ross S
        • Oandasan I.
        Process and outcome evaluation of a CBME intervention guided by program theory.
        J Eval Clin Pract. 2020; 26: 1096-1104
        • Gold M
        • Helms D
        • Guterman S.
        Identifying, monitoring, and assessing promising innovations: using evaluation to support rapid-cycle change.
        Commonwealth Fund, New York2011: 12
      4. Hargreaves MB. Rapid evaluation approaches for complex initiatives: mathematic policy research, 2014 [Internet]. 9 Nov 2020. Available at: https://aspe.hhs.gov/report/rapid-evaluation-approaches-complex-initiatives. Access 10 November 2022.

        • Hall AK
        • Rich J
        • Dagnone JD
        • et al.
        It's a marathon, not a sprint: rapid evaluation of competency-based medical education program implementation.
        Acad Med. 2020; 95: 786-793
        • Patton MQ.
        Qualitative research and evaluation methods: integrating theory and practice.
        4th ed. SAGE Publications, Thousand Oaks, Calif2015
        • Braun V
        • Clarke V.
        Using thematic analysis in psychology.
        Qual Res Psychol. 2006; 3: 77-101
        • O'Connor C
        • Joffe H.
        Intercoder reliability in qualitative research: debates and practical guidelines.
        Int J Qual Methods. 2020; (Accessed 10 November 2022): 19https://doi.org/10.1177/1609406919899220
        • Soleas E
        • Dagnone D
        • Stockley D
        • Garton K
        • van Wylick R.
        Developing academic advisors and competence committees members: a community approach to developing CBME faculty leaders.
        Can Med Educ J. 2020; 11: e46-e56
        • Tomiak A
        • Braund H
        • Egan R
        • et al.
        Exploring how the new entrustable professional activity assessment tools affect the quality of feedback given to medical oncology residents.
        J Cancer Educ. 2020; 35: 165-177
        • Ross S
        • Huie M
        • Schipper S.
        Use words that count: a content analysis to identify words and phrases that commonly appear in effective formative feedback.
        in: Presented at the International Conference on Residency Education. Vancouver, BC, 2015 (October 22–24)
        • Chan TM
        • Sebok-Syer SS
        • Sampson C
        • Monteiro S.
        The Quality of Assessment of Learning (Qual) score: validity evidence for a scoring system aimed at rating short, workplace-based comments on trainee performance.
        Teach Learn Med. 2020; 32: 319-329