VR Clinical Simulation (Unity + Meta Quest)
A world-space symptoms panel, four clinical buttons, and a headset that refused to cooperate.
A Unity XR teaching scene where a learner stands in a room, reads a holographic symptoms card, and works through History & Exam, Investigations, Explore Anatomy and Diagnosis. Most of the build time went into the invisible plumbing: which canvas the panel lived under, why the avatar never rendered in the headset, why controller rays could hover nothing, and how to pin the whole clinic to a real spot in the room with a spatial anchor.
Tools I used
How it started
It started with a panel that was already under the Intro scene and still would not show up — and with a colour picker that refused to open.
What actually happened
The scene was supposed to be simple: a symptoms card floating in front of a learner, four buttons, done. Instead the first hour went to a panel that existed in the hierarchy of the Intro scene but never appeared in play mode, a Color field on the Image component that would not surface a picker, and a Rect Transform that ignored a 1000x600 size because the canvas around it was in the wrong render mode. Then the headset added its own set of problems: the avatar was there in the editor and absent through the lenses, and the four buttons lit up perfectly with a mouse but were dead under a controller ray. The fix for the buttons turned out to be a single wrong component — EventSystem was running Input System UI Input Module, which knows about pointers and keys and nothing about tracked devices. Swapping it for XR UI Input Module and adding a Tracked Device Graphic Raycaster to the canvas made every button respond. The last piece was making the clinic stay put: a spatial anchor so the panel belongs to the room instead of drifting with the player.
Moments that changed the build
- Panel is under Intro scene but not rendering
- Image Color field shows no picker
- Resize panel to 1000x600 in Rect Transform
- Avatar not seen in headset
- Buttons work with mouse, dead with controller ray
- Remove Input System UI Input Module, add XR UI Input Module
- Add Tracked Device Graphic Raycaster to SymptomsCanvas
- Add a spatial anchor so the scene stays fixed in the room
What broke
The four buttons — History & Exam, Investigations, Explore Anatomy, Diagnosis — were fully functional in the editor and completely unclickable in VR, because the EventSystem had the desktop input module attached.
What I learned
In VR, nothing about the UI is automatic. A canvas needs a render mode, a raycaster that understands tracked devices, and an input module built for rays — three separate components that all have plausible-looking desktop equivalents.
The ebook from this project
- Read it
The Panel That Would Not Appear
Building a VR clinical simulation, one invisible component at a time
Ebook · $9.99