The Evolution of a LEAD Question
By Chris Lundeen, ABFAS Director of Examinations
In January 2022, ABFAS launched the Longitudinal Education and Assessment for Diplomates (LEAD) program to replace the traditional, once-every-ten-years recertification and self-assessment examinations. Similar to continuous certification programs introduced by other certifying boards, the LEAD program provides continuous learning that reinforces knowledge of surgical concepts and promotes retention with ongoing performance feedback.
Each quarter, Diplomates have the opportunity to answer thirty multiple-choice questions, based on topics that are current and relevant to foot and ankle surgery. After answering each question, Diplomates see the correct answer, along with complete journal citations and an educational rationale that explain why each of the incorrect options are wrong and the correct response is correct. Learning occurs as Diplomates consider the best response to the question, see the correct answer, review the rationales, and explore the journal citations. The development of quality assessment questions, rationales, and citations is foundational to the LEAD program.
The People Behind LEAD Questions
Approximately 80 ABFAS Diplomates volunteer as subject matter experts (SMEs) to write, review, and edit LEAD questions. Experienced question writers—along with the ABFAS staff and a psychometrician—train new writers on the many topics important to the development of high-quality LEAD questions, including best practices on constructing questions, image quality standards, and educational rationales. The newly trained writers then work together in a practical question-writing lab to apply what they have just learned.
After their initial training, new writers work with a mentor on all LEAD questions for their first two years. ABFAS also supports writers with question-writing guides, journal access, an image library, and additional training sessions throughout the year.
Drafting LEAD Questions
Twice a year, ABFAS staff assign each writer a task to write 5–10 new questions, along with educational rationales and supporting citations from peer-reviewed journal articles. The tasks specify a topic and whether the question should be related to foot or RRA surgery. Within the scope of that task, the writer is free to write any question they like.
Inspiration for new LEAD questions may come from a writer's personal practice, grand rounds at hospitals, or conferences. They look for learning concepts that encapsulate a Diplomate’s real-world experience and are supported by current and relevant journal articles, surgical texts, or updated guidelines.
Once the writer has completed the question, they assign a more detailed subtopic to the question to help ABFAS staff manage the balance of content in the pool of LEAD questions.
Reviewing LEAD Questions
Next, a group of SMEs trained in reviewing assessment questions examines, edits, and approves the edited questions. The questions go through two rounds of review and editing by two independent groups of three reviewers. A new group of at least five reviewers then assesses the questions for overall quality by considering relevancy to the subject category, accuracy of the medical terminology, and the quality of any associated images.
Questions that have moved forward through each stage of the review process are then ready for the LEAD assessment. Questions that are not approved may go to another group of reviewers for rework, or they may be returned to the writer with notes explaining what the writer must do to improve the question.
Once the questions are in the LEAD assessment, ABFAS monitors the questions’ statistical performance and gathers feedback from Diplomates via the LEAD system. If a potential issue is identified through these mechanisms, a group of SMEs reviews the question, along with its performance data and Diplomate feedback. If the SME group confirms a problem, then ABFAS takes appropriate corrective action.