XR (VR/AR) Interaction Flow

An XR interaction flow maps how a person enters, moves through, acts in, and exits a 3D experience — headset VR, passthrough mixed reality, or phone/headset AR. It adds dimensions a 2D flow never has: the user's body position, physical comfort, real-world safety, and the transition between reality and the rendered scene. The evidence for the constraints lives in Virtual Reality Design (comfort, locomotion, input maturity) and Augmented Reality Design (anchoring, coaching, safety); this file covers how to sequence them into a flow.

When to use

Designing any VR/MR/AR experience longer than a single view: training, review of 3D data, spatial tools, games, AR maintenance or commerce.
Adding an XR mode to an existing desktop/mobile product.
Auditing an XR app where users get lost, sick, or quit at setup.

What it answers

How does the user get in — and, just as important, safely and gracefully out?
How do they move, aim, select, and manipulate at each step, with which input (controllers, hands, gaze+pinch, rays)?
Where in the sequence do comfort and safety checks live?
What happens when tracking fails, the boundary is crossed, or the user takes the headset off mid-task?

What it includes

Entry & calibration

Pre-entry: expectation setting on a 2D surface (what you'll need, seated/standing, space required) before the headset goes on.
Boundary/guardian confirmation; seated vs. standing mode choice.
Calibration steps as explicit flow states: height/floor level, eye or hand calibration, and for AR the coaching sequence ("move your phone slowly", "find a flat surface") plus tracking-recovery guidance (Augmented Reality Design: both Apple and Google emphasize coaching UI here).
Gradual transition into immersion — fade or growing portal, not a hard cut (Meta's mixed-reality transition patterns).

Spatial navigation

Locomotion choice is a flow decision: teleport/snap-turn for general audiences; smooth locomotion only as an opt-in with vignette and constant velocity (Virtual Reality Design — acceleration is the sickness trigger).
In MR/AR, don't relocate the user at all: content moves, the room doesn't.
Orientation and re-centering: a recenter action reachable at every step (users drift, start crooked, or hand the headset over); wayfinding cues when the task spans locations (landmarks, guide lines, audio beacons).

Selection & manipulation

Map each step to its input: gaze+pinch (visionOS's primary model), controller rays for distant targets, direct grab within reach, hand tracking for coarse gestures only (Virtual Reality Design: poor for fast/precise/ occluded poses; no haptics — compensate with audio/visual confirmation).
Oversized targets, strong hover/focus states, dwell tolerance for jitter; controller/hand parity so the flow works with either.

UI surfaces

Spatial (world-anchored, diegetic) menus for in-task actions near their objects; 2D floating panels for dense text, lists, settings — placed 1–3 m out, within the comfortable gaze cone, never head-locked (Virtual Reality Design/Augmented Reality Design ergonomics).

Comfort & safety checkpoints (scheduled, not hoped for)

Flows must schedule comfort: natural break points every few minutes of sustained interaction, no sustained mid-air arm work ("gorilla arm"), rest poses between manipulation-heavy steps, and a pause/recenter state reachable from anywhere (Virtual Reality Design fatigue and comfort constraints).
Safety states: boundary crossing pauses the experience; real-world hazards are never obscured (Augmented Reality Design); headset-removed = auto-pause and resumable state.

Exit & cross-device continuity

Explicit save/exit path; gradual return to passthrough/reality; a post-session 2D summary (what you did, what's saved) since in-headset memory of detail is poor.
Desktop/mobile/VR mode transitions: define what state carries across (the annotation made in VR appears in the web app), where each mode is entered from, and a non-XR fallback for the core task where feasible (Augmented Reality Design onboarding guidance).

How to build it

Write the task flow first as if it were 2D (Task Flow) — the goal doesn't change because it's spatial.
Assign each step a place (where the user is), a pose (seated, standing, reaching), and an input.
Insert the fixed frame: entry/calibration at the front, exit/save at the back, recenter + pause reachable from every state.
Add comfort checkpoints wherever pose or locomotion is sustained.
Add failure states: tracking lost, boundary crossed, headset off, battery/thermal warnings, hands out of view.
Prototype in-headset early — paper can't test comfort.

Common mistakes

Calibration and coaching skipped or crammed into one modal; the user starts mis-scaled and lost.
Locomotion chosen for drama, not comfort; no comfort options menu.
Head-locked UI; menus behind the user; text at 2D sizes.
No recenter action; no pause; boundary events treated as crashes.
Manipulation marathons with no rest poses.
One-way door: entering XR loses the work done on desktop, or exiting loses the work done in XR.

Worked example: 3D review & annotation session

Launch (2D) → space check → Enter Scene (fade in)
  → Calibrate Height/floor → Tutorial Panel (skippable, 2D panel)
  → Teleport to Object → Select (ray or gaze+pinch)
  → Inspect (grab/turn within reach; orbit for large objects)
  → Annotate (spatial marker + voice note; text via 2D panel)
  → Save (visible confirmation) → next object or Exit (fade out)
  → 2D summary: annotations synced, shareable link

Failure branches: tracking lost → dim scene + coaching → resume at
same state; boundary crossed → pause; headset removed → auto-save,
resume prompt on return; hands not tracked → prompt controllers.
Comfort: recenter on the wrist menu; suggested break after ~20 min
or N objects; all manipulation at waist–chest height.

Domain examples (short)

VR training module: brief → demo (watch) → guided attempt (steps gated, errors coached) → free attempt (scored) → debrief in 2D. Comfort: seated option, teleport between stations, break between modules.
AR maintenance overlay: scan equipment → anchor recognized → step-by-step spatial labels with leader lines → confirm each step → log completion. Safety: brief interactions, never occlude moving parts; non-AR PDF fallback.
3D drone-map review: open orthomosaic/3D model → scale/orient (tabletop vs. immersive) → teleport between flagged issues → measure → annotate → export report to desktop.
Spatial design tool: 2D plan import → walkthrough at true scale → swap materials via world-anchored palette → snapshot views → exit to desktop editor with changes synced.

Checklist

Pre-entry expectations set on 2D; seated/standing both supported
Boundary confirmed; calibration and AR coaching are explicit steps
Gradual enter/exit transitions (fade/portal, no hard cuts)
Locomotion comfort-first (teleport/snap default; options menu)
MR/AR: user never relocated; real hazards never obscured
Recenter and pause reachable from every state
Each step has an assigned input; controller/hand parity; big targets with strong focus states
UI 1–3 m out, in the gaze cone, never head-locked
Comfort checkpoints scheduled: breaks, rest poses, no sustained arm raising (cross-ref Virtual Reality Design)
Failure states: tracking lost, boundary, headset-off (auto-save)
Work syncs across desktop/mobile/XR; non-XR fallback where feasible
Accessibility: seated/one-handed modes, captions, remappable input (Virtual Reality Design XAUR notes); tested in-headset with the SSQ when in doubt

Sources

Meta Horizon OS developer design guidelines (developers.meta.com/horizon/design) — comfort, locomotion, hand tracking, mixed-reality transitions and scene understanding.
Apple — visionOS Human Interface Guidelines; HIG Augmented Reality (developer.apple.com/design) — gaze+pinch, UI placement, coaching.
Google — ARCore UX design guidelines (developers.google.com/ar/design).
Jerald, J. (2015). The VR Book. ACM Books — comfort and adaptation.
LaViola, Kruijff, McMahan, Bowman & Poupyrev (2017). 3D User Interfaces: Theory and Practice (2nd ed.). Addison-Wesley — selection/manipulation/travel technique taxonomy.
Kennedy, R. S. et al. (1993). "Simulator Sickness Questionnaire." Int. Journal of Aviation Psychology, 3(3).
W3C — XR Accessibility User Requirements (XAUR); XR Association developers guide, accessibility chapter (xra.org).