This proposal outlines a group presentation on transforming traditional medical exams into Multiple Choice Questions (MCQs) using AI technology, focusing on two key areas. The first segment explores the technical aspects of using AI, such as Large Language Models, to generate MCQs and quality distractors. The second segment addresses critical validity factors like reliability, fairness, and efficiency in the new exam formats, with examples from the Medical Council of Canada Qualifying Examination and the new Physiotherapy Competency Exam. The presentation aims to offer insights into modernizing exam formats and ensuring robust assessment integrity, proposing a collaborative framework for educators and policymakers in the AI age.