The Future is Now: Virtual and Mixed Reality in Medical Education

POSTED: March 21st, 2025
POSTED IN: EM Pulse Q1 2025,

Written by: Timothy Koboldt, MD, FACEP
University of Missouri – Columbia

Experiential learning, particularly simulation, has already become a mainstay in medical school and residency training for physicians. The future of this learning method has already begun to arrive. Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (XR) have transitioned from science fiction to accessible technology available for purchase online, running on consumer-grade hardware. Existing technologies allow us to recreate virtual environments, simulate clinical scenarios, facilitate remote collaboration and learning, and teach concepts through both instructor-driven and learner-driven experiences. These advancements enable continuous incremental improvements in both the technology and processing power behind VR, as well as in our teaching methodologies and approaches. The foundation has been laid, and now we can build upon it, offering increasingly realistic and engaging experiences to a wide range of learners.

Defining Reality

Most people are at least familiar with VR, but it may be helpful to differentiate it from augmented reality and mixed reality. It is easiest to think of these as existing along a continuum, with VR on one end and AR on the other. In general, VR applications create completely virtual environments, isolating users from real-world inputs. When wearing a VR headset, users interact exclusively with virtual stimuli.

In contrast, AR overlays virtual elements onto the real world. This could be likened to the “Terminator” vision from the classic movie or an Instagram filter for the younger generation. AR is often—but not always—used for procedural or surgical training.

Mixed reality (XR) combines real-world and virtual elements into a seamless experience. A prime example is the Vimedix AR trainer by CAE, which overlays ultrasound images and anatomical structures onto a mannequin torso, helping learners visualize 2D ultrasound images within the context of 3D anatomy. While there is often some overlap among these distinctions, if you’re unsure which type of reality you’re experiencing, calling it XR is usually a safe bet.

Current Applications and Use Cases

The past few years have seen an explosion of healthcare-related XR applications. The ability of XR to present experiences in three dimensions at scale is difficult to describe without using a headset. Unlike a 2D video game or animation, XR offers an immersive three-dimensional experience, making it an ideal tool for teaching anatomy and radiology.

Organon VR, one of the most popular early applications, allows learners to explore human anatomy in immersive 3D. Similarly, Medical Holodeck enables users to view real cross-sectional CT or MRI imaging in virtual reality.

Simulated patient encounters and virtual simulation mannequins represent some of the most exciting and relevant applications of XR. Both individual and team-based simulations are available, with instructor-driven formats similar to traditional high-fidelity mannequin simulations, as well as asynchronous, self-paced, or AI-driven experiences. Each approach offers distinct advantages.

Team-based simulations foster collaborative learning and integrate seamlessly with existing simulation-based education. At our institution, we have incorporated XR into our simulation station rotations using the SimX platform. Here, a group of learners participates in a simulated patient encounter in real-time within a VR environment, guided by an instructor operating the case on a PC. This format retains the familiar framework of simulation education while modernizing the delivery method.

Individual and asynchronous applications provide flexibility in scheduling and offer personalized feedback. Applications like Health Scholars VR enable learners to lead a computer-generated team through high-stakes ACLS or PALS scenarios using only voice commands. While not perfect, the voice recognition in these applications is more advanced than that of typical digital assistants and better reflects the reality of team leadership during resuscitations. Standardization and repeatability allow for more meaningful feedback and performance tracking over time.

An unexpected but powerful benefit of XR technologies is their ability to support remote or asynchronous learning. For example, in the event of a pandemic that restricts large gatherings, these technologies can facilitate remote collaboration or allow small groups to engage in the same learning experiences at different times and locations.

Future Applications

If the future is now, it is exciting to imagine what comes next. Advances in computing hardware will continue to yield faster and smaller processors, making standalone and wireless headsets more powerful. The evolution of related technologies may have an even greater impact. Cloud computing and ultra-high-speed connectivity will further untether headsets from computers—eliminating immersion-breaking cables.

Haptic feedback combined with mixed reality has the potential to revolutionize procedural training. If simulations can accurately replicate resistance, force feedback, and the tactile experience of performing procedures, learners will be able to develop muscle memory for fine motor skills.

On the software side, an open-sandbox platform could enable instructors—without advanced coding skills—to create simulation cases and modules. These could run synchronously, allowing real-time adaptations, or asynchronously, providing learners with scores and feedback. Such a platform would allow instructors to share and exchange cases, reducing duplication of effort and promoting the dissemination of high-quality educational content. Future XR applications should also incorporate voice recognition and hand tracking to minimize the need for complex controller-based interactions.

Although it may take time to reach the fully immersive level of the holodeck or the OASIS from Ready Player One, I look forward to leveraging existing and emerging technologies in the short and medium term to train the medical professionals of the future.