Surgical training systems worldwide are undertaking reforms to incorporate competency-based education. There is little published evidence on how competency-based surgical training curricula can be effectively implemented and evaluated.

We describe an evaluation of the impact of the re-design of the Australian Orthopedic Association national surgical training program, which transitioned from a time-based apprenticeship model to a competency-based model in 2017. We used a mixed-methods research approach with a combination of quantitative analysis of in-training portfolio data and qualitative thematic analysis of survey responses and focus group transcripts.

Survey responses from trainees, recent graduates and training faculty revealed alignment in most areas but divergent opinions in some. Detailed discussion of these discrepancies with the focus groups enabled better understanding of the impact of the re-designed program on them and other interested parties. A national core curriculum, programmatic assessment with workplace-based tools, a formal teaching syllabus and appropriate supervision were identified as strengths of the new program. Variability in the delivery of several aspects of the curriculum and use of assessment tools were noted as ongoing challenges.

Trainee exposure to teaching and learning of professional skills was variable, as was clinical experience in all orthopaedic pathology domains. Assessment of competence in clinical and professional skills was perceived to be onerous for both trainees and trainers. A better supported faculty development process and enhanced engagement with all participants was recommended to improve buy-in to the educational aims of the program and improve assessment.

The program evaluation panel concluded that many of the original aims of the re-design process were implemented successfully but several challenges emerged, echoing common issues encountered globally in organizational change projects in healthcare and the implementation of competency-based training.