Temporal anti-aliasing and reconstruction

Follow subpixel jitter, motion reprojection, history rejection, and temporal upscaling through Asterune's renderer.

Temporal anti-aliasing (TAA) combines samples from different frames to stabilize edges and detail smaller than a pixel. Asterune offsets the camera projection by a fraction of a pixel, renders the scene, and reprojects the previous resolved image into the current view. The resolve decides how much history each pixel can retain before it writes the next image and history.

In the frame pipeline, the temporal pass runs on high-dynamic-range (HDR) scene color after world composition and glow. Motion blur, depth of field, exposure, and tone mapping follow it. Interface rendering occurs later, so screen interfaces do not enter the world image's temporal history.

Temporal resourcesCopy link

The resolve reads scene color, reverse-Z depth, motion, color history, and depth history. It draws one full-screen triangle into 3 attachments: resolved color, next color history, and next depth history. Two history sets alternate by frame index; the pass reads the opposite set from the one it writes.

These formats describe the single-sample temporal path:

Depth history and scene depth serve different purposes. Reverse-Z depth selects the nearest visible surface, with larger values nearer the camera. The motion attachment supplies clip W for the history depth comparison. For a perspective projection, clip W measures distance along the view axis.

Color history stores C / (1 + max(C.r, C.g, C.b)), rather than unbounded HDR color. The shader expands the resolved result with C / max(1 - max(C.r, C.g, C.b), 0.0001). Compression reduces the influence of bright HDR values during blending and neighborhood clipping. Exposure is applied later and therefore does not become part of this stored color history.

Subpixel projection jitterCopy link

Native-resolution TAA repeats an 8-position jitter sequence. Each position stays within half a pixel of the pixel center. The renderer converts the selected pixel offset into a projection translation using 2 * jitter.x / viewport_width and -2 * jitter.y / viewport_height. The negative Y scale accounts for texture coordinates increasing downward.

Rasterization uses the jittered projection, while motion uses the unjittered current and previous projections. A stationary object under a stationary camera therefore reports zero motion even while its coverage changes across the jitter sequence. Including jitter in motion would reproject the sample pattern instead of accumulating its different coverage samples.

Temporal rendering also biases texture gradients toward finer mip levels. The gradient multiplier is 2^(-0.5 + log2(render_scale)); native-resolution TAA uses a mip bias of -0.5. A lower render scale adds its own bias so reduced-resolution rasterization can still supply texture detail for reconstruction.

The jitter sequence advances when a frame is committed. A held, settled view uses zero jitter. Multisampled scenes and views with temporal accumulation disabled also use zero jitter.

Motion reprojectionCopy link

World vertices carry current and previous clip positions. The current position uses the current model and unjittered camera matrices. The previous position uses the previous model and camera matrices. Skinned geometry can also supply the previous vertex positions, so deformation contributes motion independently of object translation.

The fragment shader divides each clip position's XY coordinates by W, subtracts previous from current, and converts the result into displacement in target UV coordinates. The resolve then samples history at history_uv = uv - motion. Motion channels B and A retain previous and current clip W for depth handling. If either clip W is nonpositive, the world shader writes zero displacement and marks the previous depth as unknown.

Asterune dilates motion across a 3 × 3 neighborhood by taking the motion of the nearest surface in that neighborhood. This gives foreground edges the motion of nearby foreground geometry instead of the background they partially cover. A difference greater than 0.5 pixels between the pixel's own motion and the selected motion marks a moving edge. Such a pixel cannot use the relaxed history rules for still surfaces.

Sky pixels derive motion from previous_view_projection * inverse(current_view_projection) when the motion target does not already contain sky motion. When particles need a motion pass, the renderer writes sky motion first and blends particle motion over it. Water also writes motion after the opaque pass. These paths prevent later scene composition from retaining an unrelated opaque surface's motion.

History color uses 5 linearly filtered texture samples with cubic reconstruction weights around the reprojected position. This reconstructs detail more sharply than a single bilinear history sample. The history sampler clamps at texture edges, and a separate viewport test prevents clamped samples from being accepted as valid reprojection outside the previous view.

Disocclusion and moving depthCopy link

A disocclusion occurs when a moving foreground surface uncovers background. Asterune detects it from depth history, with an additional record of whether the old surface moved. Each pixel stores its own current clip W, negated when its motion reaches 0.01 pixels on either axis.

The resolve examines the 4 depth-history texels in the bilinear footprint of the reprojected history position. It ignores a texel whose interpolation weight is less than 0.01. It also ignores nonnegative depth: a stationary thin surface missing from one jittered frame has not necessarily uncovered the background. Only a previously moving surface contributes to this test.

For the remaining texels, the nearest stored depth is compared with the selected surface's previous clip W. The relative nearness is 1 - nearest_history_depth / previous_surface_depth. A smooth transition starts at 0.04 and reaches full rejection at 0.12. The result becomes a lower bound on the current frame's blend weight.

The distinction between selection and storage preserves thin geometry. Motion comes from the nearest surface in the 3 × 3 neighborhood, but depth history keeps each pixel's own depth. Storing the neighborhood's nearest depth everywhere could make a jittered wire appear to occupy pixels that it only borders.

Neighborhood clipping and still detailCopy link

History must fit the current neighborhood before it can contribute color. The native-resolution shader compresses the 9 current RGB samples, converts them to YCoCg, and computes their mean and standard deviation. YCoCg separates luminance from 2 chrominance components. The initial clipping box extends one standard deviation from the mean on each component.

The shader clips history toward the box center. It scales the history-to-center vector until the vector fits inside every component's bounds. This preserves the direction of the color difference instead of clamping each color component independently.

A still pixel receives a wider box to preserve accumulated subpixel detail. A pixel qualifies as still only when all these conditions hold:

• Motion is less than 0.01 pixels on both axes.
• The pixel is outside a moving edge.
• Disocclusion is zero.
• The reflection reactive value is less than 0.1.

The wider box reaches up to 2.5 standard deviations, bounded by the neighborhood's observed minimum and maximum. The shader also learns a luminance spread in the second depth-history channel. It updates that spread at a rate of 0.1 and widens the box by twice the previous learned spread. The learned difference is bounded by the neighborhood's luminance range, so a uniform lighting change cannot teach an arbitrarily wide clipping box.

Real lighting changes need a separate test because surface motion remains zero. The shader compares history luminance with both the pixel and neighborhood mean. If the smaller difference exceeds 3 * learned_spread + 0.04, it treats the pixel as changed and uses the tighter box. This lets a moving shadow or changed light replace an accumulated image of a stationary surface.

A history sample more than 2 box half-widths from the center also resets accumulation confidence. The distance test adds a 0.015 floor to each half-width, so a nearly uniform neighborhood does not classify every small fluctuation as a large change.

Confidence, reflections, and fresh samplesCopy link

Color-history alpha tracks accumulation confidence. A still pixel whose history passes rejection gains 1 confidence unit per frame. Motion, a moving edge, disocclusion, a reactive reflection, or detected color change resets confidence to 1. The base current-frame weight is clamp(1 / confidence, 0.015, 0.1). Thus a qualifying stable pixel progressively retains more history, while the ordinary moving-pixel blend uses 10% current color.

Screen-space reflection composition supplies a reactive signal in scene alpha above 1. The resolve extracts clamp(scene_alpha - 1, 0, 1). Reflections can move independently of the surface that receives them, so surface motion alone cannot reproject them correctly. A fully reactive reflection forces at least 30% current color and prevents classification as still.

Disocclusion and the reactive signal can raise the blend weight above its base value. If reprojection leaves the viewport, the shader takes 100% current color and resets confidence. When the current-frame weight exceeds 0.3, native TAA also reconstructs current color from the 3 × 3 neighborhood. Exponential weights account for the current jitter offset; a smooth transition increases filtering until the weight reaches 0.9. This reduces the raw coverage noise exposed when history is unavailable.

A frame with invalid history takes a separate initialization path. Native TAA copies current HDR color to the output, seeds compressed history, and initializes confidence to 1 / 0.015. The normal motion, rejection, and disocclusion tests govern subsequent frames.

Temporal upscalingCopy link

When the render rectangle is smaller than the display rectangle, the temporal pass selects its upscaling shader instead of native TAA. The source scene occupies a reduced rectangle in the full-size targets, while the output and histories remain at display resolution. The shader maps each output pixel back into that render rectangle.

Reduced-resolution rendering uses a longer Halton jitter sequence, with bases 2 and 3. The phase count is ceil(8 / render_scale²). At a scale of 0.5, the cycle contains 32 phases instead of 8. More phases supply different subpixel samples for the larger reconstruction ratio.

The upscaling shader collects a 3 × 3 source neighborhood around the jittered source position. It computes a weighted color reconstruction and the strongest sample coverage. Stable history gains confidence in proportion to this coverage, so an output pixel weakly sampled by the current jitter does not accumulate confidence as quickly as a directly covered pixel.

Motion is converted from render-rectangle UV displacement to display-rectangle displacement. The shader uses the same nearest-surface selection, moving-depth disocclusion, reflection reactivity, and cubic history sampling as native TAA. Its neighborhood box spans twice the standard deviation. The upscaling path does not apply native TAA's learned-spread expansion or still-surface lighting change test.

Moving pixels reconstruct the current frame with a separable 4 × 4 Lanczos filter and use a base current-frame weight of 0.25. The shader clamps that reconstruction to the central 2 × 2 color range to limit ringing. Invalid history initializes from bilinear source color with confidence 1. Reprojection outside the viewport also uses bilinear color; partial disocclusion blends the reconstruction toward bilinear color before temporal blending.

Tone mapping later applies Robust Contrast Adaptive Sharpening (RCAS) at strength 0.8 when the frame resolved temporally or upscaled. Sharpening reads the center and 4 axial neighbors after their tone-map curves. Its result is display color, so sharpening does not feed back into temporal history.

History lifecycle and held viewsCopy link

A newly created temporal state starts with invalid history. Resizing replaces the temporal targets and invalidates history. Toggling temporal accumulation calls reset_temporal_history, which also clears the previous camera matrix and invalidates the retained scene-color input for indirect gathering and the screen-space lighting history. A place change calls the same reset.

Camera motion updates reprojection and resets the still-frame counter. The renderer compares the unjittered camera matrices exactly; it does not apply a camera-cut distance or angle threshold in this temporal state. Ordinary camera motion therefore preserves history for per-pixel validation rather than causing a whole-image reset.

Studio can request a held view after 24 still frames. With valid history and a hold request, the shader copies the settled history instead of accumulating another jittered scene image. The projection stops jittering as well, keeping later depth-dependent effects stable across interface redraws. Camera movement releases the effective hold by resetting the still-frame count.

World edits restart settling. note_world_change(true) marks history stale while an edit continues, but keeps blending during that change. The first subsequent note_world_change(false) invalidates the stale history once, allowing the settled image to rebuild from the final scene.

Secondary interface world views temporarily pause the main view's temporal state. They render without temporal jitter, resolve, or screen-space history. Afterward, the renderer restores the main camera matrices, frame index, and history-valid flags. A secondary image therefore does not advance the main view's jitter sequence or become its previous frame.