XR frames and tracking spaces
Follow OpenXR frame pacing, predicted poses, eye-image ownership, tracking transforms, and room-scale collision.
Asterune's headset runtime uses OpenXR to schedule stereo frames and locate tracked devices at a predicted display time. The engine renders each eye into an image owned by the XR runtime. The desktop window receives a separate copy of an eye image rather than owning the headset's presentation clock.
The headset path must coordinate 3 kinds of state: the compositor's frame lifecycle, GPU access to swapchain images, and the transform from physical tracking space into the simulated world. A correct desktop camera matrix alone does not establish those relationships.
Runtime-selected graphics device
The XR runtime participates in creating the Vulkan instance and device and selects the physical GPU associated with the headset. Asterune exposes that adapter to wgpu and adopts the Vulkan handles. This keeps rendering and the compositor on the graphics connection required for shared eye images.
The engine requests the runtime's recommended stereo image dimensions and chooses a supported sRGB 8-bit swapchain format. Each eye has its own swapchain and wrapped wgpu textures. The image lifetime remains tied to the XR session; the wrapper is not an independently allocated copy of the runtime's image.
Shutdown respects that ownership. Image views, wrappers, and other objects created from the session are released before the session. The Vulkan device and instance outlive the session that uses them. A desktop surface and its mirror image have separate ownership from those eye swapchains.
Session state and the frame clock
Runtime events control whether frames can flow. A READY event begins the
stereo session; STOPPING ends it. EXITING, LOSS_PENDING, or instance loss
requests exit. The engine also tracks presence, refresh-rate changes, reference
space changes, visibility masks, and controller interaction-profile changes.
For a running session, xrWaitFrame supplies predicted display time, predicted
period, and should_render. The engine locates eyes, head, and controller input
against that display time before advancing the game frame. Simulation and
presentation preparation then execute before xrBeginFrame starts eye work.
This places simulation between the compositor wait and the rendering interval.
A false should_render skips eye rendering, but it does not cancel the frame
lifecycle. The engine still pairs begin and end and submits an empty layer list.
Missing valid eye poses takes that empty-frame path as well. An invisible frame
therefore cannot leave a pending wait unmatched.
The renderer disables its ordinary background submission handoff while a
headset owns presentation. Eye submission must stay ordered with swapchain
release and the frame's xrEndFrame call. The ordinary desktop handoff is
explained in Frame pipeline and resources.
Predicted poses and asymmetric eye projections
Eye location requires valid orientation and position flags and at least 2 stereo views. Head location has its own validity checks. Invalid velocity flags produce zero linear or angular velocity rather than treating uninitialized velocity data as a measured movement.
Each eye uses the runtime's pose and 4 field-of-view angles. Their tangents produce an off-center projection; the left and right half-angles need not be symmetric. The projection uses an infinite far plane with reverse-Z depth, matching the scene's depth convention.
The view matrix is the inverse of the eye's world pose. The engine submits the same eye poses and fields of view that it used to draw. Substituting a later pose only in the submitted layer would make the compositor interpret an image using a different camera from the one that generated it.
View-independent preparation uses a head-centered view wide enough for both eyes. Per-eye draws still use their own frusta, camera uniforms, and depth. Shadow preparation, environment resources, particles, and exposure reuse work across the stereo frame. See Anti-aliasing and render resolution for the multisample path and the single-history limitation of scene TAA.
Eye-image acquisition and release
For each eye, the engine acquires a swapchain image and waits until the runtime allows drawing. It installs that image as the renderer's offscreen target, records the eye's world and interface work, and submits the resulting commands. The image is then released to the runtime.
The final overlay also restores the exact swapchain image to color-attachment use. Interface drawing can use a linear alias of an sRGB image; the runtime must receive the original image with the intended graphics state. Releasing a wrapper does not replace that transition.
After both eyes are released, xrEndFrame submits a projection layer containing
their poses, fields of view, and swapchain rectangles. The engine then presents
the desktop mirror. The mirror copies the left eye into an owned image and
prefers mailbox or immediate presentation when available. Copy rate limiting
keeps spectator-window work from being required on every stereo frame.
Visibility masks can write hidden lens regions into depth before ordinary scene passes. They are requested when the session starts and again after the runtime reports a mask change. Each eye retains its own mask and projection. The mask changes where the engine shades; it does not change the submitted field of view.
Tracking space and world transforms
The session prefers STAGE, then LOCAL_FLOOR_EXT, then LOCAL reference space.
Those choices differ in whether the origin supplies a floor reference. The
engine maintains a separate play-space position, rotation, and scale to place
tracked meters inside the game world.
For a tracked point p, world position is
origin + rotation × (p × studs_per_metre). Tracked orientation composes with
the play-space rotation. Moving the world origin therefore preserves relative
head and hand placement without modifying the runtime's tracked poses.
Recentring records a neutral horizontal position and heading. The placement transform maps that neutral head to the requested world camera origin. Floor space includes the measured eye height in the neutral point; a local reference space uses its own head-relative vertical origin.
Automatic scale compares avatar eye height in studs with measured eye height
in meters and bounds the result from 1 to 12 studs per meter. Invalid heights
or an eye height below 0.5 m use the default scale of 1 / 0.28 studs per meter.
Height measurement requires a focused session without an explicit absent-user
report. Seated mode uses the default standing-height assumption for scale and
adjusts placement separately.
Room-scale body following and head collision
Room-scale following requires a living, unseated character outside platform-stand mode, the default character-follow camera, and a focused session that is rendering and has not reported absence. A script-controlled camera takes a separate placement path. Frames without those conditions do not supply a room-scale motor offset.
Physical walking supplies a horizontal offset from rendered character feet to the tracked head. The offset is bounded to 2 studs for one movement resolution. The character motor resolves that displacement against world collision rather than teleporting the body directly to the tracked head.
The play-space origin compensates for the physical displacement accepted by the motor. Otherwise, body following would apply physical walking a second time to the camera. Artificial body movement remains in the origin update and can contribute to the comfort vignette. Teleports reset the accumulated room movement and head-collision state instead of sweeping back toward the earlier room placement.
With the default character-follow camera and FadeOutViewOnCollision enabled,
head tracking remains live when the tracked head crosses virtual geometry.
A separate comfort calculation sweeps a small box from the last safe head
position, slides along blocking surfaces, and checks an outer overlap shell.
It uses 0.10 m inner and 0.20 m outer extents, converted to world scale.
A blocked inner region fully obscures the image; the outer shell increases the fade before an eye enters the surface. Sweeping from the last safe point also handles a head that crosses completely through a thin wall between frames. The obstruction clears geometrically when a safe route returns, and the visual fade recedes at a bounded rate. This changes visibility without forcing the tracked head pose away from the person's actual movement.
For script-facing poses, tracking state, and recenter operations, see VRService.