How it works · write-up

Plan: Capture & render wall tiles (masked/transparent + unmasked/opaque) in the GPU sprite-stream window

xenondoc/plan-wall-tiles-render.md · 21 KB · updated 2026-09-17

Context

RENDER_VIEW_SPRITE_STREAM reconstructs a frame purely from g_drawCommandStream, which today only captures object draws (ship/enemies/bullets/particles) via a hook on the object list's shared drawProc dispatch (DrawCommandStream_OnInstructionFetch, src/drawCommandStream.c:296). drawCommandStream.h's own doc comment flags the gap explicitly: "Does NOT cover background/wall tile drawing, which is a separate mechanism (handled elsewhere, later)." This plan is that "later" — adding wall tiles (both transparent/masked and opaque/unmasked) to the same reconstructed stream, so replayed frames include the level geometry the ship flies through, not just the moving objects drawn on top of it.

Scope: wall tiles only — the 16x16 masked/unmasked tile graphics at ST region 0x59c42, registered as SPRITE_REGION_USED_TILES in g_spriteMemoryRegion[] (src/xenonRender.c:187, maxSpriteCount = 337). The separate 320x384 pre-rendered "background mosaic" (SPRITE_REGION_TYPE_BACKGROUND at 0x6989c, xenonRender.c:189) that fills in empty (non-wall) tilemap cells is explicitly out of scope — those cells render as the window's black clear color for now (see "Deferred" below).

What's already in place (verified by reading the code)

Atlas side needs no changes to its output, but the g_isMasked[] source should improve. RenderUsedMaskedAndUnmaskedWallTiles (xenonRender.c:2227, already run today by the existing one-time region scan triggered by opening the "Masked Sprite" debug window) walks the manually-curated g_isMasked[] table (loaded from the initalGMask string constant via ApplyInitialMaskIfNeeded, xenonRender.c:473, originally populated by hand via mouse-click toggling in the debug UI, see ToggleMaskedAndPrint/SpriteMemoryRegions_ProcessMouseClick) and for each of up to 337 tile codes computes:

stTile = tileGraphicsBase + (uint16_t)code * 0x10 + 2*8;   // xenonRender.c:2268-2270

(the + 2*8 i.e. +16 is flagged in-source as a "TODO manual fix, otherwise first two rows are rendered as garbage" — treat this offset as an opaque, already-working constant, not something to re-derive). It then calls either DrawSTScreenBlockUnmasked_ARGB8888 (unmasked path) or DrawSTMaskedTile16_ARGB8888 (masked path), both of which internally call RecordRenderedSprite keyed by exactly that stTile address, tagged SPRITE_FORMAT_4PLANES_TILE (unmasked, 8 bytes/row straight copy — xenonRender.c:925-952) or SPRITE_FORMAT_4PLANES_TILE_MASKED (masked, 12 bytes/row AND/OR/data-long triplet — xenonRender.c:1775-1808). This already exports every individual wall tile into the atlas (spritecatalog.bin), and SpriteAtlas_FindRect (keyed by ST address) already resolves it. No changes needed to this scan's output or the atlas format.

g_isMasked[] — cross-check against live ST memory instead of trusting it blindly. g_isMasked[] exists purely so this offline scan can walk the packed tile-graphics blob sequentially: masked tiles occupy 12 bytes/row (12 "code units") and unmasked occupy 8 bytes/row (8 "code units") in that blob (CalculateNextCode, xenonRender.c:2207-2222), so the scan needs to know each slot's format before it can find the next one — that's a real, separate need from what the live capture hook requires. The live tilemap word already carries its own mask flag directly (code & 0x8000, no lookup needed) — bit15 in the actual level data is the "ST memory source of truth" for whether a given placed tile is masked, it's just not organized as a lookup-by-index table anywhere; it's baked per-placement into the level's tilemap. That means the new live capture hook (below) can, as a side effect, record every distinct (index, isMasked) pair it observes from real bit15 flags during play, and log a diagnostic if that ever disagrees with g_isMasked[index] — turning g_isMasked[] from "hand-toggled and trusted" into "hand-toggled and continuously cross-checked against the game's own runtime data," with a path to eventually regenerating initalGMask wholesale from a captured playthrough instead of mouse-clicking (a single session won't necessarily observe all 337 indices, so keep the hand-curated table as fallback for whatever isn't observed). See "Implementation" step 2 below.

Live draw routine (from prior RE this session). Wall tiles are drawn once per frame by ST_DrawGeneratedBackgroundFromTileMap_FUN_00001dd0 (0x1dd0-0x21ab), called from the main loop before the object/bullet drawing pass on the same freshly-swapped buffer. It fully redraws the visible 20-column tile grid every frame (no incremental update), with the masked/unmasked/ empty dispatch fully inlined and triplicated across three row-bands (top partial, 11 full middle rows, bottom partial), each doing, per tile column:

uint16_t code = *tilemapCell++;      // bit15 = masked flag
if (code == 0)         { /* empty cell -> background mosaic, OUT OF SCOPE */ }
else if (code & 0x8000) { /* masked: index = code & 0x7fff, 12 bytes/row - matches
                             DrawSTMaskedTile16_ARGB8888's layout */ }
else                    { /* unmasked: index = code, 8 bytes/row - matches
                             DrawSTScreenBlockUnmasked_ARGB8888's layout */ }

The three fetch sites are at PCs 0x1ea8 / 0x1f2a / 0x214e (move.w (A1)+,D0, reading the raw tile code); the instruction immediately after each (0x1eaa / 0x1f2c / 0x2150) is where D0 holds the just-fetched code and A0 holds the destination screen address for that tile's top-left pixel.

The "fine-scroll source-variant" read on wall tiles is unconfirmed — a hypothesis, not an established fact. An earlier draft of this plan asserted (by analogy with the already-RE'd bullet preshift trick) that a variant-selector mechanism handles smooth sub-tile horizontal scrolling for wall tiles. That's plausible but unverified — and there's a competing explanation: some wall tiles (and enemies attached to walls) are animated, and a small per-tile "pick one of N pre-stored variants" selector is exactly what animation-frame selection would also look like from the decompiled code alone. Before committing to either interpretation, check whether the selector value (i) tracks the level's horizontal scroll offset (supports the sub-pixel-shift reading) or (ii) advances on its own periodic/frame-based cadence independent of scroll position (supports the animation-frame reading) — it may even be both, for different tiles. This determines whether a fix (if wanted) belongs in the vertex builder (a scroll-position offset) or the atlas (extra frame variants, same shape as how bullets' preshifted copies were added). Deferred either way for this plan — see "Deferred" below.

Vertical position must not be assumed to snap to a 16px grid. This is not just caution — there's a concrete, if unverified, lead already sitting in this file as dead code: RenderMiddleTileMap_WithMaskedTiles_Debug (xenonRender.c:2340-2469, #if 0'd out, marked "unused") reads a 16-bit value at ST address 0x00000CCE (labeled yScroll there, confirmed present via direct read of that dead-code block) and folds it into the tilemap base pointer used to walk tile codes — i.e. there's at least one prior, unverified attempt at treating vertical positioning as scroll-dependent rather than fixed. That code is explicitly marked speculative ("Approximation... if the result is visibly shifted/wrong, trace A4 for one tile") so it's a lead to re-investigate later, not a confirmed mechanism now. The good news: this plan's A0-destination- decode approach doesn't need to resolve this ambiguity to be correct. y = byteOffset / 160 is computed directly from wherever the game itself wrote (A0), on a per-scanline basis — it is never quantized to 16px by our code, regardless of what drives the game's own vertical placement. Risk #2 below is the empirical check: if captured y values turn out to only ever be multiples of 16, the vertical-scroll concern is moot for capture purposes; if they aren't, this approach is already handling it correctly by construction (unlike a hypothetical grid/counter-based approach, which is exactly why A0-decode was chosen over that alternative).

flowchart LR
  subgraph "per frame, main loop"
    A["ST_DrawGeneratedBackgroundFromTileMap\n(0x1dd0), 3 row-bands"] -->|draws tiles to buffer| B["object drawProc dispatch\n(0x3ee4, existing hook)"]
  end
  A -.->|NEW hook, 3 fetch PCs| C[DrawCommandStream_OnTileDrawInstructionFetch]
  B -.->|existing hook| D[DrawCommandStream_OnInstructionFetch]
  C --> E[g_drawCommandStream]
  D --> E
  E -->|ForEachInFrame, push order = capture order| F["SpriteStreamVertexBuilder_AppendEntry\n(screentrace.c:411)"]
  F --> G["single SDL_DrawGPUPrimitives call\ntiles drawn first -> objects on top"]

Design decisions

Reuse the existing pipeline — no new stream/struct/shader. A wall tile draw fits DrawMaskedSpriteEntry exactly: spriteId = the computed stTile, x/y = tile screen position, atlas rect via SpriteAtlas_FindRect, isFlash = false. objectId isn't a real object here, so it gets a synthetic per-cell value purely so debug tooling has something stable to key on. Because tiles are drawn before objects in the game's own per-frame order, pushing tile entries in capture order gives correct z-order for free via the existing DrawCommandStream_ForEachInFrame walk (oldest-first) feeding the single vertex array / single draw call — no sorting, no second draw call.

New sibling capture function, not an extension of the existing one. DrawCommandStream_OnInstructionFetch (object dispatch, drawCommandStream.c:296) stays untouched. Add DrawCommandStream_OnTileDrawInstructionFetch(uint32_t addr) — different PCs, different register extraction, no drawProc/no-op-stub logic needed.

Screen position from the destination pointer (A0), not a reconstructed grid counter. Deriving col/row from an external per-frame counter would assume every row-band always processes exactly 20 columns, which the partial top/bottom bands make risky. Decoding A0 directly is immune to that: the live ST low-res screen is 320x200, 4 bitplanes interleaved, 160 bytes/scanline (already relied on elsewhere in this codebase, e.g. xenonRender.c:664,693), 8 bytes per 16px column, so:

byteOffset = A0 - drawBufferBase;
x = ((byteOffset % 160) / 8) * 16;
y = byteOffset / 160;

drawBufferBase is read live via STMemory_ReadLong(DRAW_BUFFER_BASE_ADDR) (the existing 0x000406 constant, currently only used to detect a write there in DrawCommandStream_OnMemoryWrite — this is a new read of the same address, both plain ST addresses so the subtraction is valid). Note y here is a direct per-scanline value (byteOffset / 160, no rounding to 16) — it is not assumed to land on a 16px grid, per the note above; whatever exact row the game wrote to is what gets captured.

Implementation

src/drawCommandStream.c / src/includes/drawCommandStream.h

1. New constants alongside the existing DRAWPROC_DISPATCH_PC block: - TILE_GRAPHICS_BASE_PTR = 0x0004F008u — read the tile graphics base live via STMemory_ReadLong(TILE_GRAPHICS_BASE_PTR) rather than hardcoding 0x59c42. This pointer (confirmed live-RE'd earlier this session as PTR_DAT_0004f008, and independently confirmed present at xenonRender.c:2363-2364's dead debug code) is exactly what the game's own A4 register is loaded from before it walks tile graphics — reading it live is both more robust (survives if a different level/build relocates the blob) and removes one more hand-verified magic constant from this hook. Cross-check once at implementation time that the live value equals 0x59c42 (the SPRITE_REGION_USED_TILES entry's stAddr in g_spriteMemoryRegion[], xenonRender.c:187) to confirm this is really the same base the atlas was built from; if it ever isn't, prefer the live value and treat the hardcoded atlas-side constant as the one needing a second look. - TILE_FETCH_PC_TOP = 0x00001eaau, TILE_FETCH_PC_MID = 0x00001f2cu, TILE_FETCH_PC_BOTTOM = 0x00002150u — the post-fetch PCs where D0/A0 are both valid (matches the DRAWPROC_DISPATCH_PC convention of hooking the instruction after the value-loading one, same as A1 being already-loaded at the existing JSR hook).

2. New cross-check accessor in xenonRender.c (the g_isMasked[] live cross-check):

// xenonRender.c, next to g_isMasked[]/ApplyInitialMaskIfNeeded -- declared in screentrace.h
// so drawCommandStream.c can cross-check live-observed mask bits against the hand-curated table.
bool XenonRender_CheckIsMaskedTileIndex(int index, bool observedIsMasked)
{
  ApplyInitialMaskIfNeeded();
  if (index < 0 || index >= maxRenderedSprites)
    return true;   /* out of range -- nothing to compare against, don't flag */
  return g_isMasked[index] == observedIsMasked;
}

Called from the new hook below purely for diagnostics — never gates whether an entry gets pushed (the live bit15 is authoritative for drawing; a mismatch just means g_isMasked[] is stale for that index and should be revisited, e.g. by regenerating initalGMask from a fuller captured playthrough).

3. New function:

void DrawCommandStream_OnTileDrawInstructionFetch(uint32_t addr)
{
  uint16_t code;
  uint32_t index, tileGraphicsBase, stTile, a0, drawBufferBase, byteOffset;
  int16_t x, y;
  uint16_t atlasX, atlasY, atlasW, atlasH;
  bool isMasked;

  if (addr != TILE_FETCH_PC_TOP && addr != TILE_FETCH_PC_MID && addr != TILE_FETCH_PC_BOTTOM)
    return;

  code = (uint16_t)m68k_dreg(regs, 0);
  isMasked = (code & 0x8000u) != 0;
  index = code & 0x7fffu;
  if (index == 0)
    return;   /* empty cell -> background mosaic, out of scope */

  if (!XenonRender_CheckIsMaskedTileIndex((int)index, isMasked) && !AlreadyLoggedSpriteId(0x00D00000u | index))
    printf("DrawCommandStream: tile index=%u live isMasked=%d disagrees with g_isMasked[] -- table is stale for this index\n",
      index, isMasked);

  tileGraphicsBase = STMemory_ReadLong(TILE_GRAPHICS_BASE_PTR);   /* PTR_DAT_0004f008, read live */
  stTile = tileGraphicsBase + index * 0x10u + 2u*8u;   /* mirrors xenonRender.c:2268-2270 exactly */

  a0 = m68k_areg(regs, 0);
  drawBufferBase = STMemory_ReadLong(DRAW_BUFFER_BASE_ADDR);
  byteOffset = a0 - drawBufferBase;
  x = (int16_t)(((byteOffset % 160u) / 8u) * 16u);
  y = (int16_t)(byteOffset / 160u);   /* NOT rounded to 16 -- see note above */

  if (!SpriteAtlas_FindRect(stTile, &atlasX, &atlasY, &atlasW, &atlasH))
  {
    if (!AlreadyLoggedSpriteId(stTile))
      printf("DrawCommandStream: tile stTile=0x%06X (index=%u) NOT found in atlas. x=%d y=%d\n",
        stTile, index, x, y);
    return;
  }

  DrawCommandStream_PushMaskedSprite(&g_drawCommandStream,
    0xFFFF0000u | ((uint32_t)y << 9) | (uint32_t)x,   /* synthetic objectId, unambiguous vs real ST addrs */
    stTile, x, y, CyclesGlobalClockCounter, atlasX, atlasY, atlasW, atlasH, /*isFlash=*/false);
}

Reuses the existing AlreadyLoggedSpriteId dedup table as-is, keyed by stTile for the "not in atlas" case and by a distinguishable 0x00D00000 | index key for the mask-mismatch case (same table, same one-shot-per-key behavior, no changes needed to the table itself).

4. Call the new function from the same per-instruction-fetch chokepoint that already calls DrawCommandStream_OnInstructionFetch(addr) (src/screentrace.c, ScreenTrace_LogRead): c DrawCommandStream_OnInstructionFetch(addr); DrawCommandStream_OnTileDrawInstructionFetch(addr);

5. Update the header doc comment in drawCommandStream.h (the "Does NOT cover background/wall tile drawing" sentence) to describe the new hook, mirroring the existing doc style/detail level for DrawCommandStream_OnInstructionFetch.

6. Declare XenonRender_CheckIsMaskedTileIndex in src/includes/screentrace.h (alongside the existing RenderedSpriteCatalog_SaveToFile declaration) so drawCommandStream.c can call it — matches how that header already exposes select xenonRender.c functions to other translation units.

No atlas, vertex-builder, GPU, or shader changes

SpriteAtlas_FindRect, SpriteStreamVertexBuilder_AppendEntry (screentrace.c:411), the GPU pipeline, and both shaders are already format-agnostic — they only care about stAddress -> atlas rect and x/y/atlasRect -> quad — so wall tiles flow through unchanged once they're in the stream.

Known risks / verify empirically during implementation

(Same "capture, log anomalies, iterate" pattern already used for bullets this session — expect at least one round of this.)

1. Formula correspondence: does live-captured index reliably resolve via SpriteAtlas_FindRect(tileGraphicsBase + index*0x10 + 16, ...)? This hinges on the hand-curated g_isMasked[] table's tile addresses lining up 1:1 with what the live tilemap's index actually means. The "not found in atlas" diagnostic is the detection mechanism; the new mask-mismatch diagnostic (step 3 above) is a second, independent signal for the same underlying question.
2. A0-to-x/y decode: confirm captured x values always land on a 16px boundary (0, 16, ..., 304). Do not apply the same expectation to y — per the note above, log the distribution of captured y values across a session with the level actively scrolling and check whether they're ever not a multiple of 16. Either outcome is informative: all-multiples supports "horizontal-only scroll, no special vertical handling needed"; non-multiples confirm real vertical fine-positioning exists and (per the A0-decode design) is already being captured correctly without extra work.
3. Scroll vs. animation: log the fetched code's masked/unmasked flag and (if a variant selector can be located) its value per index over consecutive frames, for a tile known to be animated. A value that changes on a fixed cadence regardless of horizontal scroll speed points to animation frames; a value tracking scroll offset points to sub-pixel shift. Needed before any follow-up work picks a fix for the "Deferred" scroll-smoothness item below.

Deferred (explicitly out of scope here)

• Empty tilemap cells (background-mosaic fill) — render as black for now. Follow-up needs per-cell atlas export of the background mosaic (currently one big atlas rect for the whole image) plus a capture-side sub-rect lookup.
• Sub-tile fine positioning/animation — this plan's x is quantized to the 16px column grid (by construction: 8 bytes/column in the destination address decode), and any variant-selector mechanism (whether it turns out to be horizontal sub-pixel shift or tile animation, risk #3 above) is not reproduced. y is not quantized (see risk #2) so no vertical follow-up is needed unless investigation says otherwise. Once risk #3 is answered, the fix is either a per-frame vertex-time X offset (if real sub-pixel scroll) or additional atlas variants selected by an observed frame index (if animation, same shape as bullets' preshifted copies).

Verification

1. Build (no atlas/.NET regeneration needed — the atlas already has tile entries from the existing one-time scan; just make sure the "Masked Sprite" debug window has been opened at least once first, same precondition as today).
2. Open the "Sprite Stream" debug window during actual gameplay (not idle) so the ship flies over real level geometry; confirm wall tiles appear as a background layer behind the ship/enemies (columns aligned to the 16px grid; rows may or may not be, per risk #2), with masked tiles showing transparency and unmasked tiles fully opaque. Also confirm tiles positioned partially or fully off-screen (e.g. top/bottom partial row-bands) simply don't appear rather than glitching — this should fall out of standard GPU clip-space rasterization for free (confirmed: SpriteStreamVertexBuilder_AppendEntry, screentrace.c:411, does no CPU-side bounds clamping on x/y before handing coordinates to the vertex shader), so no separate row-band handling should be needed.
3. Watch stdout for the new "tile not found in atlas" and "disagrees with g_isMasked[]" diagnostics — ideally silent; if either fires, use it to debug risks #1 and #3.
4. Confirm object entries (ship/enemies/bullets, already working) still render on top of the new tile layer — validates the draw-order-from-capture-order assumption.