Deepthi's Reading

Clinical

061

The Panel That Would Not Appear

Building a VR clinical simulation, one invisible component at a time

Unity · Meta Quest / OpenXR · XR Interaction Toolkit · Meta Building Blocks

Edition #061 · 4 chapters · 15 min read

The Panel That Would Not Appear

Building a VR clinical simulation, one invisible component at a time

A holographic symptoms card, four clinical buttons, and a headset that showed none of it. This edition follows the real debugging path through Unity XR: canvases, render modes, avatars, controller rays and spatial anchors.

Everything in this edition comes from one build session on a Unity VR teaching scene. No generic XR tutorial content — only the components that actually broke, in the order they broke, with the beginner explanation I needed at the time.

Start reading Back to the library

Start here

What you'll understand by the end

Not mastery — understanding. By the last page you'll be able to explain these in your own words, even with no technical background.

•Why a UI panel can exist in a scene and still be invisible
•What World Space canvas render mode actually changes
•Why an avatar renders in the editor but not in the headset
•Why controller rays cannot click ordinary Unity buttons
•What a spatial anchor is and what it is anchoring to

The concept ladder — the order I actually learned things

A scene is a container, not a guarantee that something renders

↓

A Canvas decides where UI lives: on the screen or in the world

↓

A Rect Transform in world space is measured in Unity units, not pixels

↓

A camera decides what the headset sees

↓

A raycaster decides what can be pointed at

↓

An input module decides what counts as a click

↓

A spatial anchor decides where the world stays

Chapter 01

It is already under the Intro scene

Track 1 · My journey

The panel was there. I could see SymptomsPanel in the hierarchy, nested under the Intro scene, exactly where the instructions said to put it. And it did not appear. My first assumption was that I had added it to the wrong scene, so I checked twice, then a third time. The scene was right. What was wrong was one level up. A UI element does not render because it is in a scene; it renders because a Canvas above it knows how to draw it, and that Canvas has a render mode. Mine was still set to Screen Space, which in a VR project means it is drawn onto a screen that a headset never shows you. The other symptom lined up too: I was trying to set the Image colour to a dark blue with alpha around 180 so the card would read as a hologram, and the Color box was not giving me a picker the way the instructions described — because I was clicking on a component that was not the one holding the visible graphic. Switching the Canvas to World Space changed the meaning of every number underneath it. The Width 1000 and Height 600 I had typed into the Rect Transform stopped being pixels on a screen and became a slab of geometry standing in the room. That is when the panel finally existed somewhere I could walk up to.

Track 2 · Learn with Deepthi

Canvas render mode (Screen Space vs World Space)

USED — evidence shows I used it

What did I just discover?: That my panel was in the right scene and still invisible, because the Canvas above it was drawing to a screen the headset never shows.
What I thought it meant: I thought a Canvas was just a folder for UI elements.
What I learned: The Canvas is the thing that renders; the panel is only content inside it.
What is it, really?: A component that converts UI elements into geometry, either overlaid on a camera output or placed as a flat object in 3D space.
Think of it like this (analogy): A cinema screen. Screen Space is a screen strapped to your face. World Space is a screen bolted to the wall that you can walk around.
How does it actually work?: You set the render mode; the Canvas then interprets its children's Rect Transforms either as pixels on the display or as units of a physical slab in the scene.
Why does this exist?: Because most games need flat HUDs, and only some need UI you can physically approach. Unity supports both from one component.
Why did it matter to my project?: Because in VR there is no flat screen to pin a HUD to — every readable card has to exist somewhere in the room.
Where I used it: On SymptomsCanvas, the parent of the SymptomsPanel holding the four clinical buttons.
Common beginner misunderstanding: That adding UI to a VR scene automatically makes it visible in VR.
What I got wrong: I debugged the panel for a long time before I looked at its parent Canvas.
What this tool cannot do: World Space UI still needs a raycaster and an input module before anyone can interact with it — rendering and clicking are separate problems.
What surprised me: How completely the same numbers changed meaning: 1000x600 went from pixels to a wall-sized slab the instant I switched modes.

What happens behind the scenes

What the tool is doing

Unity converts the Canvas hierarchy into renderable geometry and places it according to the chosen render mode.

What I am doing

I chose World Space and set the panel size and colour so it reads as a translucent hologram card.

What data is moving where?

1.I set render mode on the Canvas
2.Unity re-interprets child Rect Transforms as world units
3.The XR camera renders the canvas geometry like any other object
4.The headset displays it as a physical card in the room

One thing to remember

Remember this

A UI element renders because of its Canvas, not because of its scene.

Try it yourself

Take any panel, switch its Canvas between Screen Space and World Space, and watch what the same width and height do.

Learning checkpoint

Before this chapter

I thought putting an object in the correct scene was enough for it to show up.

After this chapter

I check the Canvas before I check the object.

What changed my mind

Parenting is not rendering. The Canvas, not the scene, decides whether UI exists in the world.

Can you answer these?

?Where does a UI element get its render mode from?
?Why did 1000x600 behave differently after switching to World Space?

Chapter 02

The avatar nobody could see

Track 1 · My journey

The next thing missing was the avatar. In the editor it stood there, fully visible, doing nothing wrong. Put the headset on and the room was empty. This one taught me that "the scene" and "what the headset renders" are two different questions. The headset does not show you the editor viewport; it shows you whatever the XR camera under the camera rig can see, filtered by culling, layers, and scale. An object can be perfectly present and still be behind the near clip plane, on a layer the XR camera does not draw, or standing at a scale that puts it either inside your face or a hundred metres away. I stopped trying to fix the avatar and started interrogating the camera instead: which camera is actually active in play mode, what is it allowed to see, and where is the rig placed relative to the thing I expect to look at. That reframing — debug the viewer, not the viewed — is the single most useful habit this project gave me.

Track 2 · Learn with Deepthi

What the headset camera can actually see

TESTED — I experimented with it

What did I just discover?: That an object can be fully present in the scene and still never reach the headset display.
What I thought it meant: I thought "it is in the scene" and "I can see it" were the same statement.
What I learned: The headset view is one camera with its own clip planes, layer mask and rig position.
What is it, really?: The rendered image is the result of a specific XR camera filtering the scene by distance, layer and frustum.
Think of it like this (analogy): A security monitor. The room is full of people; the monitor only shows whoever is inside that one camera's cone.
How does it actually work?: The XR camera under the rig computes what falls inside its frustum and layer mask each frame, and only that gets drawn to each eye.
Why does this exist?: Rendering everything in a scene would be wasteful, so every camera is told what it is responsible for.
Why did it matter to my project?: Because in VR you cannot casually orbit the editor view to find a missing object — you have to reason about the camera.
Where I used it: While debugging why the avatar was invisible through the lenses but fine in the editor.
Common beginner misunderstanding: That the editor Scene view is a preview of what the headset shows. It is a different camera entirely.
What I got wrong: I kept adjusting the avatar instead of interrogating the camera.
What this tool cannot do: Camera reasoning explains invisibility, not every VR rendering problem — shaders, materials and scale can still betray you.
What surprised me: How often "missing object" really means "wrong frame of reference".

What happens behind the scenes

What the tool is doing

The XR camera culls by clip plane, layer mask and frustum, then renders per eye.

What I am doing

I checked which camera was active, where the rig sat, and what layers it drew.

What data is moving where?

1.Rig places the XR camera in the room
2.Camera culls the scene by distance and layer
3.Remaining geometry renders once per eye
4.Headset displays the two images

One thing to remember

Remember this

When something is invisible in VR, debug the viewer before the viewed.

Try it yourself

Move an object 5cm from your headset camera and watch it disappear behind the near clip plane.

Learning checkpoint

Before this chapter

I assumed the headset shows me the same thing the editor does.

After this chapter

I treat the headset view as one specific camera with its own rules.

What changed my mind

The question changed from "why is the avatar missing" to "what is this camera allowed to draw".

Can you answer these?

?Name two reasons an object can be present in a scene and invisible in a headset.
?Which camera renders what you see in the headset?

Chapter 03

Four buttons, one wrong component

Track 1 · My journey

The four buttons — History & Exam, Investigations, Explore Anatomy, Diagnosis — worked. With a mouse. Click, highlight, response, everything. In the headset I could point a controller ray straight at them and nothing happened. No hover, no highlight, no click. The whole failure came down to one component on the EventSystem: Input System UI Input Module. It is the default, it looks correct, and it understands pointers and keyboards. It does not understand a tracked device floating in space. I removed it and added XR UI Input Module, which exposes Click Action, Move Action, Point Action, and tracked device position and orientation — the fields that tell Unity a controller pose is a pointer. That was half of it. The canvas also needed to be pointable. A Graphic Raycaster handles ordinary screen-space hit testing; a Tracked Device Graphic Raycaster is what makes a world-space canvas respond to XR rays. With both added to SymptomsCanvas, the rays from the ray interactors under the camera rig started hovering and clicking exactly like a mouse would. Two components. Hours of thinking the buttons were broken.

Track 2 · Learn with Deepthi

XR UI Input Module and Tracked Device Graphic Raycaster

USED — evidence shows I used it

What did I just discover?: That my four buttons were never broken; nothing in the scene was allowed to point at them.
What I thought it meant: I assumed a Unity button responds to any pointer, including a VR ray.
What I learned: VR interaction is a chain, and every link is a separate component you have to add on purpose.
What is it, really?: XR UI Input Module reads tracked device pose and actions and feeds them to the EventSystem; Tracked Device Graphic Raycaster lets a world-space canvas be hit by those rays.
Think of it like this (analogy): A doorbell. The button works fine; the problem was there was no wire from the gate to the bell.
How does it actually work?: The ray interactor casts from the controller pose, the tracked-device raycaster reports which UI graphic the ray hits, and the XR UI input module converts the trigger action into a click event on that graphic.
Why does this exist?: Because the default input module was written for pointers and keys, long before tracked devices existed as first-class input.
Why did it matter to my project?: Because without it, every VR menu you build is decorative.
Where I used it: On the EventSystem and on SymptomsCanvas, for History & Exam, Investigations, Explore Anatomy and Diagnosis.
Common beginner misunderstanding: That removing Input System UI Input Module breaks desktop testing permanently — the XR module handles the tracked path, and you plan your test setup around that.
What I got wrong: I spent time editing button transitions and colours, convinced the buttons themselves were misconfigured.
What this tool cannot do: It only wires up UI. Grabbing, teleporting and physical interactions come from separate interactor components.
What surprised me: How small the actual fix was: remove one component, add another, add a raycaster. Three clicks after hours of searching.

What happens behind the scenes

What the tool is doing

Unity casts the controller ray, resolves the hit graphic and dispatches hover and click events through the EventSystem.

What I am doing

I removed the desktop input module, added XR UI Input Module, and added Tracked Device Graphic Raycaster to the canvas.

What data is moving where?

1.Controller reports its pose and trigger action
2.Ray interactor casts a ray from that pose
3.Tracked Device Graphic Raycaster reports which UI graphic the ray hit
4.XR UI Input Module turns the trigger into hover and click events
5.The button runs its onClick handler

What is happening right now?

Controller pose

Ray interactor casts ray

Tracked Device Graphic Raycaster finds the button

XR UI Input Module sends the click

Button responds

One thing to remember

Remember this

The button was never broken. The pointer was never allowed to reach it.

Try it yourself

Add XR UI Input Module and a Tracked Device Graphic Raycaster to any dead VR menu and test hover before you test clicks.

Learning checkpoint

Before this chapter

I thought a working button was a working button, regardless of the device.

After this chapter

I read input as a chain: device pose, ray, raycaster, input module, button.

What changed my mind

The bug was never in the UI. It was in what was allowed to point at the UI.

Can you answer these?

?Which component lets a world-space canvas receive VR rays?
?Why does Input System UI Input Module fail with a controller?

Chapter 04

A special anchor core: making the clinic stay put

Track 1 · My journey

The last problem was subtler than the others, because nothing looked broken. The panel appeared, the rays worked, and the whole scene quietly drifted — it belonged to the app, not to the room. Move around, restart, and the clinic was somewhere slightly different. A spatial anchor fixes that. Instead of placing the panel at coordinates in the app world, you ask the headset to remember a pose in the real room and attach your content to it. The headset is already building a map of the space; an anchor is a named point in that map that survives tracking corrections and, when saved, survives sessions. For this project it meant one anchor as a core: place it once at the spot where the learner stands, parent the symptoms card and the anatomy space to it, and everything else is positioned relative to that anchor rather than to an arbitrary origin. The clinic stops being a floating UI and starts being a place.

Track 2 · Learn with Deepthi

Spatial anchors

EXPLORED — I researched or discussed it

What did I just discover?: That "where things are" in VR has two possible meanings: relative to the app origin, or relative to the real room.
What I thought it meant: I assumed placing an object at fixed coordinates meant it would stay in the same physical spot.
What I learned: Tracking drifts and corrects; anchors are how content survives those corrections.
What is it, really?: A stored pose registered with the headset's own map of the space, which the runtime keeps updating as its understanding of the room improves.
Think of it like this (analogy): A nail in the wall. You can hang anything off it, and when the room shifts in your mental map, the nail moves with the wall, not with your idea of where the wall was.
How does it actually work?: The headset continuously maps the room. You request an anchor at a pose; the runtime binds it to features of that map and adjusts its transform whenever the map is refined. Your content is parented to the anchor, so it follows.
Why does this exist?: Because headset tracking is an estimate. Without anchors, everything you place inherits the drift of that estimate.
Why did it matter to my project?: Because a clinical simulation should occupy the same corner of the room every time the learner puts the headset on.
Where I used it: As the single core anchor the symptoms card and anatomy space are parented to.
Common beginner misunderstanding: That an anchor stores a position. It stores a relationship to the room's map, which is why it can move to stay correct.
What I got wrong: I thought of positioning as a coordinate problem long before I thought of it as a frame-of-reference problem.
What this tool cannot do: Anchors are room-specific. Persisting or sharing them across sessions or users involves extra platform APIs and permissions.
What surprised me: That the correct fix for drift is to move less content, not more: one anchor, everything parented to it.

What happens behind the scenes

What the tool is doing

The headset runtime maintains a spatial map and re-solves anchor poses as tracking improves.

What I am doing

I placed one anchor as the scene core and parented the clinic content to it.

What data is moving where?

1.Headset maps the physical room
2.I create an anchor at a chosen pose
3.The runtime binds that pose to the room map
4.Scene content is parented to the anchor
5.Tracking corrections move the anchor, and the content with it

What is happening right now?

Room is mapped

Anchor created at chosen pose

Content parented to anchor

Anchor stays put as tracking refines

One thing to remember

Remember this

Anchor once, parent everything, and the scene belongs to the room instead of the app.

Try it yourself

Place an anchor, restart the app, and see whether your content comes back to the same physical spot.

Learning checkpoint

Before this chapter

I placed everything relative to the app origin and expected it to hold.

After this chapter

I place one anchor and hang the scene off it.

What changed my mind

Positioning stopped being a coordinate problem and became a "which frame of reference" problem.

Can you answer these?

?What does a spatial anchor attach your content to?
?Why does anchoring survive small tracking corrections?

Appendix

Deepthi's Dictionary

Canvas: The Unity component that actually draws UI. Everything you see as a button or panel is content inside one.
World Space (render mode): A Canvas setting that places UI in the 3D room as a physical slab instead of overlaying it on the screen. Required for VR menus.
Rect Transform: The position and size component used by UI elements. In world space its width and height are Unity units, not pixels.
EventSystem: The single object in a scene that decides what is being pointed at and what counts as a click.
Input System UI Input Module: The default input translator, built for mouse, touch and keyboard. It does not understand VR controllers.
XR UI Input Module: The VR version of the input translator. It reads controller pose and trigger actions and turns them into UI events.
Graphic Raycaster: The component that works out which UI graphic a normal screen pointer is over.
Tracked Device Graphic Raycaster: The component that lets a world-space canvas be hit by rays from VR controllers or hands.
XR Ray Interactor: The component on a hand or controller that emits the pointing ray you see in the headset.
Camera rig: The hierarchy that positions the headset cameras in the scene and moves them as you move.
Culling: Unity deciding not to draw something because it is out of range, out of frustum, or on an ignored layer.
Spatial anchor: A pose the headset remembers in your real room, used to keep virtual content in a fixed physical spot.
Tracking drift: Small errors in the headset's estimate of where it is, corrected over time — the reason anchors exist.