ADR-0044: The sensor reconstructs a smaller scene at display size

2,630 words

Status: accepted · 19 Sep 2026 · amended 10 Nov 2026: reconstruction reads the optics' masked motion and the temporal layout adds reactivity alone. Supersedes the logarithmic WebGPU depth convention of ADR-0003 and the anti-aliasing renderer remount described in ADR-0029.

Context

Terrain, atmospheric shells and transparent effects shade the same display pixels. Reducing the scene target reduces that fragment work, but a bilinear resolve loses edges and detail. Temporal reconstruction also needs motion and linearizable depth. The logarithmic depth buffer preserves the scene's range but cannot be passed through the perspective depth formula FSR uses.

Anti-aliasing and reconstruction cannot be independent switches. Temporal FSR requires single-sampled input, while supersampling and reduced render scale ask the same target to grow and shrink. Rebuilding the renderer for an edge setting also destroys unrelated device resources and recompiles the scene.

Decision

The sensor owns reconstruction, one picture record owns its legal settings, and WebGPU uses reversed floating-point depth.

render/picture.ts defines aa, scale and sharpness. Native MSAA remains the default. Spatial FSR accepts raw or four-sample input at reduced resolution; temporal FSR accepts zero samples at native or reduced resolution. Supersampling uses native scale and doubles each drawing-buffer axis. The ratios are 1.5 for Quality, 1.7 for Balanced, 2 for Performance and 3 for Ultra performance. Sharpness is off, standard at 0.8, or crisp at 1. render.gamma is a display preference beside them, 0.5 to 2 with 1 as the curve is authored: a power on the displayed range up to white and on nothing above it, written to the tone curve as a uniform, because a monitor in a bright room and one in a dark room do not agree about where the midtones sit and the curve cannot know the room.

render.picture migrates saved render.aa choices once. The page's ?picture= override does not write the preference. The resolver preserves an unsupported wish while WebGL uses native scale and resolves temporal edges to MSAA. The panel disables unavailable choices and explains why. Picture changes rebuild the sensor; output-format changes still rebuild the renderer.

WebGPU uses reversedDepthBuffer and a FloatType scene depth attachment. WebGL selects logarithmic depth before backend initialization. A standard float depth buffer over the 0.05 m to 10¹⁰ m camera range loses distant separation; reversing it preserves relative precision without a fragment depth write. render/sceneLayout.ts holds the samples, depth type and one row per attachment — name, channels, storage, blend — and the scene pass, the MRT node and the warm-up stand-in are each built from those rows.

Depth convention also governs explicit depth writes and ordering. The scene's far depth is zero on reversed WebGPU and one on the logarithmic fallback; expanded line ribbons derive depth from their emitted clip-space vertex. Three r185 reverses the entire draw list for reversed depth, including authored layer order. sceneOrder.ts compensates the layer and stable-ID keys so a galaxy backdrop cannot draw over the foreground.

render/upscale.ts wraps the raw Upscaler from @pmndrs/upscaler 0.5.0. It runs once per render call on three's device, between the small scene pass and the display-size optical passes. The library compiles the configured path's pipelines asynchronously and refuses a dispatch until they exist, so the boot's upscaler step awaits them after configure names the path, and a frame drawn sooner skips the dispatch. A picture changed mid-session builds a new chain, and the chain on screen keeps drawing under its own layout until the new one's pipelines exist (scene/chainHandover.ts); swapping at once shows the zero-initialized output for as long as the compile takes. The library's GPU timer exists only while gpuTiming is on: the full timing level and a pass timeline's run turn it on, and nothing else pays for timestamp queries. Three's native velocity uses the unjittered projection. The sensor applies jitter and clears it in finally beside its renderer-state restoration. Jitter preserves the display lens aspect, including when reduced render dimensions round differently. Resizing reconfigures the working textures and resets history. Explicit camera cuts advance the presentation-only pictureEpoch; mode, staging and processing-domain changes also reset history.

The sensor remains the exposure owner. Photographic processing supplies its resolved pre-exposure and residual conditioning; Enhanced supplies unity because its separately composed sky and surfaces do not share one physical exposure scalar. FSR's own automatic meter is disabled. Reconstruction does not introduce another tone curve or change the opaque output guarantee.

The scene's temporal layout adds an RGBA8 reactive attachment and nothing else. Reconstruction's motion is the optics' motion attachment, which every layout carries: FSR reads .xy of an RGBA16F NDC delta, now minus then, which is that attachment's format and its first two channels. The one velocity node writes it, and the temporal path points that node at the kernel's unjittered projection. motion blends over by source alpha and an overlay writes alpha zero, so a flare, a plume, a warp streak or an entry trace leaves the surface's vector under it; a vector of the overlay's own would be reprojected, dilated and locked from for neighbors it never covered, and reactivity cannot repair that, because it suppresses history only where the overlay drew. The cloud composite writes motion by its opacity, so a thin cloud hands FSR the cloud's vector and the surface's in the share each holds. apps/game/src/render/upscale.gpu.test.ts holds it with a 0.5 × 0.5 overlay traveling with the eye over a surface 4 m out: after a 0.25 m pan, the bare edge and the covered center both read −0.108 of clip space.

Red carries rejection coverage; alpha carries material opacity for independent coverage blending. Opaque surfaces replace the hidden background, and translucent coverage combines as a union. Max blending retained the galaxy background's rejection underneath solid terrain, preventing accumulation. WebGPU rejects source-alpha blending on an R8 fragment output, so correct occlusion spends three extra bytes per render pixel. Opaque surfaces and starfield sprites write zero; atmosphere, clouds and rings write their alpha; water writes 0.3. Star disks, additive overlays and instrument geometry write one, including flares, plumes, warp effects and traces. The shared sensorRadiance wrapper supplies the coverage, so an overlay does not need to name a render-target layout.

The optical contract does not move with it. motion stays undilated surface motion beside reciprocal view distance, and FSR dilates its own copy. FSR's temporal guides carry closest-depth-dilated UV motion and linear depth, which do not supply the optics' contract; defocus and shutter motion do not reuse them, and a conversion would need separate compatibility measurements.

Alternatives considered

  • The library's stock node keys work to three's frame counter and owns its timing and jitter hooks. The sensor already owns each of those decisions, including repeated measurement renders in one task.
  • Feeding logarithmic depth into the perspective reconstruction formula gives a monotone value with the wrong distance, corrupting disocclusion tests.
  • A two-channel velocity attachment of reconstruction's own cannot take the overlay mask. Dawn refuses a blend that reads source alpha on a fragment output without one — srcFactor (BlendFactor::SrcAlpha) … is reading alpha but it is missing from fragment output — and three emits two components for an RG attachment; render/sceneLayout.test.ts refuses that row in Node. Masking it without the blend writes zero where the surface's vector belongs, which tells reconstruction the ground is still. Widening it to RGBA16F buys the mask that motion already carries, for 4 B a render pixel more of MRT writes.
  • A second scene render for an automatically generated reactive mask spends the scene work reconstruction is intended to reduce. An MRT attachment carries reactivity in the existing draw.
  • Reducing the terrain selection's pixel budget with render scale changes geometry detail. Selection and lens angles remain display-referred.
  • Bilinear reconstruction is a diagnostic baseline, not a player preference. Dynamic resolution and generated frames are outside this implementation.

Consequences

The native picture keeps the existing default and device lifetime. A temporal picture spends additional attachments, compute work and history storage; a smaller scene does not guarantee a faster complete frame. Performance and image acceptance use generated worlds outside Sol, at both display ratios, with a fixed camera and photographic instant. The measurements below record that acceptance; the reproduction protocol retains the comparison procedure.

ir.picture() reports actual render/display dimensions, jitter phase, reset count, initialization time and available per-pass GPU timings. workingTextureBytes counts the library's raw texture allocations. It excludes the scene and optical targets, allocation padding and driver overhead, and is not total device VRAM. The compute initialization has its own warm-up census unit. Debug views expose motion, depth, disocclusion, age, locks, exposure, shading change and reactivity. The public ir.timing('full') switch enables the upscaler's own GPU timestamps when the device supports them; off disables its query writes. Three's timestamp tracking stays off at every level: nothing reads its figure, which double-counts beside a canvas output pass, and ir.passes times every pass three encodes with a query set of its own. The upscaler lends that timeline its timer for a run, so ir.passes reports reconstruction under owned as the package's begin-to-end means. The raw upscaler's fresh-sample queue is drained each submission and does not grow with flight duration, and game.render logs the selected backend and resolved picture.

Reconstruction remains presentation state. Neither its preference, history, cut counter nor working textures enter the canonical state hash or save.

Measured preset frames

The held, shipped Tau Ceti Dusk and The Far Shore presets are measured at 1600 × 900 CSS pixels, DPR 1 and 2. Each entry is milliseconds per complete presented frame: the median of five 60-frame batches after a discarded warm-up batch, with timing instrumentation off. Detailed GPU pass timings are sampled separately. All reconstruction settings use standard sharpness; spatial rows retain four-sample MSAA.

The native and spatial pictures were measured on an Apple M5 MacBook Air with 10 GPU cores and 32 GB memory, macOS 27.0, Chrome 153.0.0.0 and Node 26.5.0, using the development server:

Preset DPR Native MSAA Spatial Quality Spatial Performance
Tau Ceti Dusk 1 3.683 3.508 3.285
Tau Ceti Dusk 2 11.252 11.022 10.640
The Far Shore 1 6.883 4.772 4.187
The Far Shore 2 22.558 18.788 13.163

The temporal pictures are measured on the layout above, motion read and reactive added, on a Mac mini (Mac18,5, Apple M6, 32 GB), macOS 27.2, Chrome 157.0.8103.40 and Node 26.13.0, against a production preview with the display asleep. Each preset and DPR is one boot with the world paused, the picture chosen in the graphics panel inside it, and native MSAA taken in the same boot as the control the temporal columns are read against:

Preset DPR Native MSAA, in boot Temporal Native Temporal Quality Temporal Performance
Tau Ceti Dusk 1 4.485 5.137 3.863 3.278
Tau Ceti Dusk 2 14.033 15.955 11.542 9.435
The Far Shore 1 7.147 6.290 4.405 3.965
The Far Shore 2 24.373 20.493 13.250 10.420

The two machines' native MSAA columns differ by 0.8–2.8 ms, so a temporal figure is compared with the native MSAA beside it, never across tables. Temporal reconstruction is not always faster. Temporal Native costs more than native MSAA on Tau Ceti Dusk at both display ratios and less on Far Shore; Temporal Quality and Performance are cheaper than native MSAA on both presets at both ratios on the Mac mini. These are held frames on two machines, not a frame-rate guarantee during flight. Every measurement preserved the renderer object and canonical state hash. On the M5, patch count and display-referred terrain selection stayed constant across picture settings at each preset and DPR; on the Mac mini, patch count did (906 and 1,227 on Tau Ceti Dusk, 777 and 867 on Far Shore, at DPR 1 and 2).

The same sixteen cells on the layout that also drew a two-channel velocity row, at bbea04da on the same machine and the same day, are 0.008–0.167 ms slower in every temporal cell, while the in-boot control moves by −0.155 to +0.043 ms between the two builds: the masked motion costs nothing and the row's removal saves at most a sixth of a millisecond, at native DPR 2. The cells, their batches and the protocol are in velocity-2026-11-10.md. Temporal Quality reports 79,668,017 raw working-texture bytes at DPR 1 and 318,712,817 at DPR 2 on both layouts there; on the M5 it reported 78,706,817 and 314,868,017.

At Tau Ceti Dusk, DPR 2, Temporal Quality's separately sampled reconstruction, exposure, shading-change, accumulation and RCAS passes took 0.459, 0.131, 0.393, 2.687 and 0.459 ms respectively. Read through ir.passes at the same preset on an Apple M5 with 10 GPU cores (Mac17,4) on 11 October 2026 — Chrome 155, two 60-frame runs in one boot, eight samples each, the most the package's eight readback slots return — they read 0.66–0.69, 0.11–0.15, 0.48–0.53, 2.88–2.98 and 0.51–0.78 ms. Each is the package's begin-to-end pair, which on this GPU is latency rather than cost: the first run put the gap between the scene pass and the optics, where the reconstruction's work falls on the timeline, at 1.01 ms. The working-texture counts above are not total device VRAM. On the M5, Temporal Quality's measured kernel initialization was 0.1–0.3 ms; that timer excludes subsequent working-texture configuration and scene warm-up.

For DPR 1 stills, RGB8 RMSE against native MSAA was 9.592 for Spatial Quality and 8.579 for Temporal Quality on Tau Ceti Dusk; Far Shore measured 3.972 and 2.999 respectively. Each comparison used the same 1600 × 870 crop, excluding the bottom 30 pixels of UI, with temporal captures held at jitter phase zero. Two boots of Tau Ceti Dusk Temporal Quality differed by 0.271 RMSE. These values describe encoded screenshot differences; they do not measure motion stability or establish a perceptual quality threshold.

A separate Chrome lifecycle run switched through eight edge and scale settings, including supersampling, Spatial Balanced and Ultra, and Temporal Balanced, Ultra and Native. The renderer, canonical hash and display terrain viewport were preserved. Instrumentation observed 58 raw FSR textures created and 58 destroyed, with none remaining after the return to native MSAA. This checks those setting transitions. A resize to 937 × 613 CSS pixels at DPR 2 produced a 1874 × 1226 display target and 1249 × 817 Quality input, while the camera aspect remained 937/613.

Runtime image checks also cover Rings Beyond the Center (far-ringrise), Centauri Daybreak (centauri-daybreak) and Under Tau Ceti III (tau-ceti-moonrise), completing the five shipped external-system presets. A 90-frame Far Shore camera pan measured 54.1 fps without isolated strobe frames. That bounded observation does not prove every motion path. A gameplay save/load at Tau Ceti Dusk's exact surface coordinates preserved its canonical hash immediately and after one paused second; that run measured 0.4 ms kernel initialization.

An isolated Chrome document forced adapter acquisition to return null, then warmed and rendered a 64 × 64 scene through the production renderer factory and sensor. It selected WebGLBackend, logarithmic depth and native resolution, with reversed depth disabled and zero FSR working textures. The saved Temporal Quality preference with standard sharpness remained intact. Only the expected fallback warning appeared. This verifies the actual fallback path on a small scene, not full-preset WebGL image acceptance.

The bilinear URL diagnostic also renders Tau Ceti Dusk at 1066 × 600 input and 1600 × 900 display with 906 patches, while the stored MSAA, native, standard-sharpness preference remains unchanged. The selected physical-GPU suite passes 79 tests across twelve files covering the sensor, optical motion, camera processing, reconstruction, reactive coverage and layered scene effects.