Introduction The Accreditation Council for Graduate Medical Education (ACGME) is the governing body responsible for accrediting graduate medical training programme in the USA. The Emergency Medicine Milestones (EM-Milestones) were developed by the ACGME and American Board of Emergency Medicine as a guide and monitoring tool for the knowledge, skills, abilities and experiences to be acquired during training. Alumni surveys have been reported as a valuable resource for training programme to identify areas for improvement; however, there are few studies regarding programme improvement in emergency medicine. We aimed to use the EM-Milestones, adapted as an alumni self-assessment survey, to identify areas for training programme improvement.

Methods This study was conducted at an urban, academic affiliated, community hospital in New York city with an emergency medicine training programme consisting of 30 residents over 3 years. Alumni of our emergency medicine training programme were sent an EM-Milestones-based self-assessment survey. Participants evaluated their ability in each EM-Milestones subcompetency on a Likert scale. Data were analysed using descriptive statistics.

Results Response rate was 74% (69/93). Alumni reported achieving the target performance in 5/6 general competencies, with Systems-Based Practice falling below the target performance. The survey further identified 6/23 subcompetencies (Pharmacotherapy, Ultrasound, Wound Management, Patient Safety, Systems-Based Management and Technology) falling below the target performance level.

Discussion Alumni self-evaluation of competence using the EM-Milestones provides valuable information concerning confidence to practice independently; these data, coupled with regular milestone evaluation of existing trainees, can identify problem areas and provide a blueprint for targeted programme improvement.