Autopilot · write-up

Level 3 gate regression: historical comparison, 3 October 2026

xenondoc/LEVEL3_GATE_REGRESSION_ANALYSIS_20261003.MD · 25 KB · updated 2026-10-05

Finding

The repair is implemented in C. Full-entry R277 reached Level 4 at frame 30214, opened every gate, lost no lives and reported no stalls. It took three 4-shield projectile hits; it is not a damage-free acceptance. The historical diagnosis below explains the original failure; implementation and validation results follow it.

The old Python controller did clear all nine gates. The C migration did not preserve its unrestricted gate-topology search and retained-route behavior. More importantly, C never modeled the original Level 3 update that renews the camera's backward allowance every frame. It uses the captured, post-camera 16-pixel allowance as a lasting bound on navigation and ship prediction.

This is a demonstrated camera-model omission, with a Level 3-specific rule to implement in the shared camera model. It is not evidence of changed terrain, an inaccessible physical passage, or a need to force movement through hazards. The omission existed in the September native runs; it was not introduced by today's G1 experiments. No gameplay source was changed during this historical investigation.

Search coverage and limits

Indexed 2,743 .x2events/.x2events.gz files: 1,828 in D:/src/hatari/xenon_tools/run_logs, 691 in E:/xenon_runs, and 224 in E:/xenon_autopilot_runs. A drive-wide E: filename search found no additional event recordings outside these two E: directories. The archive/runner-log directories were also searched. Sampling captured level and scroll fields identified 545 recordings containing Level 3 observations. This inventory is not a full gameplay replay of every file.

78 files could not be indexed by the current decoder: 66 use older protocol versions 3–5, two have an incompatible trailing payload, and ten have no canonical frames. Three empty streams have Level 3 names. The only additional unpaired Level 3 gameplay JSONL, level3-stage2-state-173.jsonl, has two frames and supplies no gate-completion evidence. These limitations are recorded rather than treating unreadable files as successful or failed runs.

The investigation used captured inputs, actual checkpoint RAM, manifests, saved source snapshots and working-tree patches. Recomputing decisions with today's DLL is not evidence of what an older pilot chose. Scroll zero and a later-level recording name are also insufficient: shop transitions expose temporary scalar values, and many final-encounter runs start with all gates already cleared.

Inventory: work/l2-source-comparison/level3-recording-inventory.json.

Proven older gate progression

Every gate consists of 22 authored cells: 14 body cells and eight cap cells. The table below reads the original big-endian tile-map words in each final checkpoint, not a controller's completion flag. Recording and checkpoint share the listed basename, under D:/src/hatari/xenon_tools/run_logs.

These are a chain of checkpoint runs, not proof of a single uninterrupted, damage-free campaign. They establish the user's recollection that the previous controller could pass all gates.

The old Python _ensure_level3_initial_route ignores camera bounds for topology, keeps a live route until the gate changes or the route ends, and keeps egress ownership through the third bend. In contrast, the initial C migration (6f602e56, “Move Level 3 gate progression and carrier intercept into C”, 13 September) uses xap_mission_route_to: searches are bounded by maximum_y, destinations are clamped, and routes refresh after 64 frames. Its documented live test covered 400 observations with no gate destruction; that validation did not establish equivalence across all nine gates.

September native source differences

Relevant older recording:

D:/src/hatari/xenon_tools/run_logs/level4-from-l3-shop-20260925-r4.validation/level4-from-l3-shop-20260925-r4.x2events

Despite its filename, it plays the Level 3 maze. Checkpoint RAM proves G1 and G2 open, G3's eight cap cells still solid, and G4 open. The remaining gates are closed. It stalls later and is not a successful all-gate native baseline.

Its saved source/native and working-tree.patch contain two changes absent from the current sources:

1. keep_route retains a gate route while its gate, destination and map revision agree and its next segment is physically free; the 64-frame refresh alone does not discard it. This appeared between R1 and R3.
2. xap_level3_gate_route_action overrides the planner with the G1 route's movement when the next physical segment is free and no checked hazard is within 48 pixels. This was the R3-to-R4 change, with one driver call.

Neither change occurs in git log --all -S searches. The manifest marks those files modified. These were uncommitted experiments, not committed fixes lost by a merge. R4 reached farther than R3, but still stalled and took four hits; blindly restoring its forced-input override would not establish safety.

The complete generated Level 3 map section is identical between R4 and current sources. Recent map-data changes belong to other levels.

Missing original camera rule

The main loop calls the level hook before the object and camera passes:

$007A96  jsr     $4F024.l
$04F024  bra.w   $4F3DE

For ordinary Level 3 maze scrolling, that hook does:

$04F418  cmpi.w  #$00D0, $0CD0.w   ; scroll <= 208 enters final-wave case
$04F41E  bgt.w   $4F460
...
$04F460  cmpi.w  #$0A30, $0CEA.w   ; backward limit must be at least 2608
$04F466  bge.b   $4F46E
$04F468  move.w  #$0A30, $0CEA.w
$04F46E  rts

The final-wave branch instead enforces a minimum backward limit of 32 at $4F422..$4F42A. The earlier first-shop boundary has its own 2800 handling at $4F3DE..$4F414. These phase boundaries must remain distinct.

The common camera code at $7B5A..$7BF6 subsequently clamps scrolling and contracts $CEA to the local scroll-plus-16 value. The next level hook restores it again. This explains why a capture can show only 16 pixels while a paused checkpoint exposes 2608, and why the ship can keep backscrolling in live play.

The main-loop call, dispatch branch, complete Level 3 rule, and common camera code are byte-identical between the August successful Python checkpoint and R258's native checkpoint. This is not a change in original game code.

xap_prepare_player_state currently provides repeated backward-limit floors for the Level 2 spider, Level 4 first boss and Level 5 final boss. It has no Level 3 equivalent. xap_maneuver_advance_ship therefore keeps shrinking its forecast limit, while mission navigation restricts maximum_y to that limit plus the player's bottom-screen coordinate.

Live input evidence in the September R4 recording: frames 22376–22440 consume Down continuously, and scroll rises from 2077 to 2129, a 52-pixel retreat. The first captured backward limit is 2093, allowing only 16 pixels according to the incomplete C model. R258 also demonstrates 26 pixels of retreat at 27662–27688 from an initial 16-pixel allowance. Respawn camera jumps are excluded from these comparisons.

Same-map route counterexamples

These diagnostics advance only C observations and map state from the recording; they do not run a Python gameplay controller. They query the same native physical graph and gate station twice, changing only the maximum world Y. The restored maximum is 2608 + 176 = 2784.

Full recent recording paths:

• E:/xenon_runs/level3-forward-full-20261003-r245.validation/level3-forward-full-20261003-r245.x2events
• E:/xenon_runs/level3-g1-route-owner-20261003-r258.validation/level3-g1-route-owner-20261003-r258.x2events

At R258 frame 28244, the C observed solid-cell counts and actual checkpoint RAM agree exactly: [0,0,8,0,22,22,22,22,22]. The same is true in September R4 at 25916. The later G3 stall is not stale destruction data. Small temporary disagreements while destruction occurs are consistent with the map's two-frame absence confirmation.

Another diagnostic error made this look like a G2 stall: choose_gate updates state->level3.gate only after route success. When selecting G3 fails, the trace retains the old G2 identifier. Actual cells prove G2 was destroyed.

Correct repair scope and acceptance

1. Model Level 3's per-update camera restoration in C ship prediction and use the same allowance for mission navigation. Preserve the shop/final-wave branches; do not relax camera limits in other levels.
2. Make failed gate selection observable as the actual selected gate, rather than leaving the last successful gate label. Do not infer progression from that label alone.
3. Recheck whether route retention still needs adjustment after the correct camera model restores legitimate replanning. Remove the unsuccessful broad pacing experiment from R259 when implementing the evidenced correction. Do not restore an unchecked movement override as the main solution.
4. Validate from before G1 through G9, plus a checkpoint before the G6/G7 backtrack. Require actual tile clearing, continued progression, no life loss, and report every shield hit. Then repeat the whole level to check the earlier passage and accepted carrier/cash tactic.

These diagnostics prove a route-bound error. They do not prove that simply restoring the camera floor will produce a damage-free live level; safety and route following still need live validation after the repair.

Reproducing the investigation

From the repository root, the reusable diagnostic scripts are:

• work/analyze_level3_history.py: recording inventory and RAM gate counts.
• work/audit_level3_camera_rule.py: original-code hashes/disassembly and actual Down intervals exceeding the first captured limit.
• work/compare_level3_gate_observations.py: C-map versus checkpoint RAM and optional same-map camera-bound route queries.

Example route comparison:

python work/compare_level3_gate_observations.py E:/xenon_runs/level3-g1-route-owner-20261003-r258.validation/level3-g1-route-owner-20261003-r258.x2events --route-frames 22721 23045 28244 --output work/l2-source-comparison/level3-camera-route-counterfactual.json

The initial camera audit used Python Capstone for original M68K instructions. ReVa subsequently confirmed the same rule in /levels/level3-memory, at $4F3DE (Level3_UpdateCameraBoundsAndFinalWaves). This is the Level 3 overlay, not a dump from another level. Evidence JSON and source diffs are under work/l2-source-comparison/level3-* and sep25-r*-to-*-*.diff.

Repair and additional live findings

The repair adds XAP_CAMERA_LEVEL3 to prepared player/maneuver state. Each C physics step restores the current phase's backward floor before player movement: 2608 above scroll Y208, 32 in the final-wave region. Mission routing uses that same allowance. The rule is derived from the captured level, so it does not require a visible boss controller. ABI 105 includes the matching ctypes layouts for replay/tests; resident gameplay remains entirely C.

The first live checks exposed separate route-execution problems:

• R260: the swarm waiting tactic used the newly available whole-maze range to choose a goal below the screen. Waiting stations now remain within the visible playfield; gate topology still has the full legitimate backscroll.
• R261/R262: a collision-free G3 retreat with 5px clearance was rejected by the ordinary 15px reserve. The maze uses its existing 4px reserve for gate travel too. This alone did not solve the encounter; both runs lost a life.
• R263: long forecast tails and periodic reconnects kept undoing retreat progress. Valid gate connectors now survive the 64-frame refresh, including stations projected onto legal hull space. Retreats use the old route controller's 12-frame reaction window, rechecked every observation. Safe stationary suffixes do not bypass reconsideration of gate travel.
• R264: no damage, but the ship repeated six Down / two Up inputs near (75,1815). At frame 23819, Right was initially legal but horizontal-only input advanced the camera; the forecast hit terrain on step three. A descending turn now keeps Down and world-row feedback active. The shortened reaction window alone was insufficient.
• R266/R267: ordinary search could choose a safe hold while rejecting every direct approach. The bounded multi-turn navigation search now also considers these blocked maze approaches, rather than running only when every ordinary candidate collides. This passed the left shaft and reached the lower crossing.
• R268: at (212,2075), the ship was inside the waypoint's X interval [212,220], but not aligned enough to turn safely around (216,2076). It requested neutral without advancing the route. At obstructed maze bends, the stopping interval now extends across the corner so a real ship step can obtain a clear onward segment. Advancing still checks the actual hull's line.

An experimental moving lookahead point (R265) did not solve the problem and was removed. No frame-number steering rule or unchecked movement override was added. The route-execution changes are confined to Level 3 maze navigation; the articulated carrier tactic and other levels keep their existing policy.

The R263 directional-projectile audit checked 3816 one-frame, 2198 eight-frame, and 1097 sixteen-frame predictions. None differed from surviving recorded objects by more than one pixel. The gate retreat failure was not an erroneous projectile direction. See work/l2-source-comparison/r263-hazard-audit.json.

Recordings for these checks are in E:/xenon_runs/, each with both AVIs and 150-frame checkpoints. Full filenames follow the directory stem:

For example, R264's event stream is E:/xenon_runs/level3-retreat-reaction-20261003-r264.validation/level3-retreat-reaction-20261003-r264.x2events.

Full-entry coverage and narrow-shaft search

R269 reached Level 4 at frame 29919. Its frame-28196 checkpoint contains zero solid cells in all nine gate groups. It resumed before G3 and took two 4-shield projectile hits (26148 and 27110), with no life loss. This proved the remaining gate sequence, including the G6/G7 retreat, but did not prove the earlier entry. Recording: E:/xenon_runs/level3-clear-corner-progress-20261003-r269.validation/level3-clear-corner-progress-20261003-r269.x2events.

R270 resumed at Level 3 entry (17268). It took hits at 18642 and 18948 and stalled at 23126. Its waiting destination was visible, but its connector dipped to world Y2496 below the current viewport. Restricting the station rectangle alone was insufficient: the whole staging connector now uses the visible maximum Y. The full maze camera allowance remains available to gate routes. Recording: E:/xenon_runs/level3-full-repaired-20261003-r270.validation/level3-full-repaired-20261003-r270.x2events.

The blocked-route beam search is now an explicit XAP_NAVIGATION_ESCAPE_PROGRESS mode, selected only for Level 3 gate and maze-exit travel. The early swarm passage retains contact-only escape search. R271 then passed the early passage, carrier and first two gates without shield loss, but stalled after G2 at frame 24560. Recording: E:/xenon_runs/level3-bounded-staging-full-20261003-r271.validation/level3-bounded-staging-full-20261003-r271.x2events.

At R271 frame 22027, DOWN's forecast meets a diagonal projectile on update 10. Four held lateral inputs hit the shaft wall after two or three updates, although a single lateral input is terrain-free. The escape search considered only the four-input moves and repeatedly chose Up or neutral, undoing the camera retreat. An initial suspicion of a camera-physics mismatch was disproved: $661C in the Level 3 ReVa program and the actual next-frame coordinates match C. It was the planner changing the held input, not the game refusing DOWN.

For maze progress only, a wall-rejected lateral move now gets one short-tap retry. It applies one lateral input, then resumes vertical route steering for the rest of that four-frame block without immediately reversing X. Complete collision checking and the existing work budget still apply. Successful held moves receive no extra retry, and other encounters keep their existing search. The synthetic narrow-shaft test checks safe passage around a real raster corner, where ordinary safe-hold selection cannot reach the destination.

R272 passed the remaining maze and reached Level 4 at 29110, without stalls or life loss. Its four remaining projectile hits were 22765, 24051, 24071 and 24783 (4 shield each). It tested the initial short-tap variant before the final vertical-only continuation refinement. Recording: E:/xenon_runs/level3-shaft-tap-retreat-20261003-r272.validation/level3-shaft-tap-retreat-20261003-r272.x2events.

R269's later directional-shot audit checked 993 one-frame, 713 eight-frame and 417 sixteen-frame paths; no surviving-object sample differed by more than one pixel. This verifies motion for those samples, not collision timing or universal avoidance. Details: work/l2-source-comparison/r269-projectile-audit.json.

Full-run counterexample: ceiling exposure and the maze exit

R273 passed the early passage and carrier without damage and opened all nine gates, but it is not an accepted full-level result. Its incidents were 22615 (4 shield), 23714 (8), 23716 (12) and 25789 (4). At the G6 approach, the ship reached screen Y16 before newly allocated enemies entered on top of it. At 23716 it overlapped both an enemy and a projectile. This was ceiling exposure, not evidence of a wrongly predicted projectile direction. Recording: E:/xenon_runs/level3-maze-repaired-full-20261003-r273.validation/level3-maze-repaired-full-20261003-r273.x2events.

Forward gate approaches now retain the 120–152 screen reaction band. Retreat legs and later low lateral crossings retain their world-space row and camera allowance; the tested initial G1 entry pacing remains unchanged.

After G9, R273 repeatedly returned from world Y580 toward Y792 along a route whose destination was (160,452). Generic screen pacing opposed the first backward leg, and the 64-frame refresh moved the destination as the ship retreated. The result was an extended loop, not an uncleared gate. Recording was stopped gracefully at 44733 and both AVIs were finalized.

The Level 3 maze exit now shares gate travel's bounded progress search and short retreat horizon. Its valid connector survives periodic refresh until reached or invalidated by changed topology, camera bounds or a blocked next segment. Screen pacing is disabled on its retreat bends. These changes are confined to Level 3 navigation; articulated-carrier and other-level tactics remain outside this progress-search mode.

Tests exercise G6's forward reaction space, G1 entry/retreat ownership and a post-G9 detour retained beyond the refresh interval and invalidated by a map revision. The current targeted suite contains 19 Level 3 mission tests and 54 player, maneuver, driver, decision, session and planner tests.

R274 avoided the G6 enemy collision and cleared all nine gates. Its checkpoint at 26154 has zero solid cells in every authored gate group. It took one 4-shield hit at 24647, but still failed to finish: its center advance target projected into the same pocket at (164,544). A connected projected point did not guarantee onward progress. It was stopped at 34102. Recording: E:/xenon_runs/level3-gate-reaction-exit-20261003-r274.validation/level3-gate-reaction-exit-20261003-r274.x2events.

For Level 3 ordinary advancement, C now searches a reachable forward region across X14–304, from max(minimum_y, player_y-128) through the next 32 pixels. This allows an exit around a blocked center point instead of projecting that point back into the pocket. The chosen connector owns world-space travel; screen pacing cannot turn its legal forward leg into a stationary mission. A raster test places a wide obstacle on the center line and requires an actual destination beyond it on either side. There are now 20 Level 3 mission tests.

R275 demonstrated the exit, but lost a life in the final radial-worm waves: 26896 (8 shield), 26897 (31 shield), life lost at 26914. The planner rejected every immediate move as wall contact after a formation hold took ownership near the right edge. The real hull remained outside terrain; the enlarged wall reserve left no checked escape. The final-wave region, identified by the original scroll boundary at Y208, now uses the existing contact escape search and real sprite hull, with the same 4px Level 3 reserve. This does not enable the progress beam for arbitrary formation holds or other levels. Recording: E:/xenon_runs/level3-forward-region-exit-20261003-r275.validation/level3-forward-region-exit-20261003-r275.x2events.

R276 resumed before those final waves and reached Level 4 at 29024, without a stall or lost life. One projectile hit at 27265 cost 4 shield. This is a focused encounter result, not a damage-free or whole-level result. Recording: E:/xenon_runs/level3-final-worm-escape-20261003-r276.validation/level3-final-worm-escape-20261003-r276.x2events.

Final full-entry validation: R277

Recording: E:/xenon_runs/level3-full-repaired-20261003-r277.validation/level3-full-repaired-20261003-r277.x2events. Start: E:/xenon_runs/campaign-three-fast-20261002-r192-1.validation/checkpoints/f17268-periodic.sav.

The official Release build rebuilt both Hatari and the replay DLL before this resident-C run. It used validate_campaign.py native-run, fast-forward, both AVIs, 150-frame checkpoints and Level-3 trace logging. Python recorded the session and inspected C output; it supplied no per-frame gameplay decisions.

The run reached Level 4 at 30214, with no lost life or recorded stall. The 28164 checkpoint has zero solid cells in every authored G1–G9 group. The early swarm passage and articulated carrier passed without shield loss. Both AVIs finalized cleanly, each with 45,334 indexed video frames and no RIFF structure/index errors. Targeted verification passed 20 Level 3 mission tests and 54 player, maneuver, driver, decision, session and planner tests.

Remaining shield incidents, all in this recording:

The first two recomputed contact frames have no verified immediate escape. That does not establish that the earlier approach was optimal. At the third hit, C reported a safe plan with 1.414px clearance, while the recorded player and projectile rectangles touched. That discrepancy still needs a collision geometry/timing audit; no projectile-direction error has been demonstrated for this hit. Diagnostics: work/l2-source-comparison/r277-damage-inspect/.

Captured pickup audit infers collection of all 10 carrier drops born at 20795 and all 20 final-encounter drops born at 28592: each retired beside the ship. There is no explicit collection event, so these are inferred outcomes. Two Rear Shot pickups were deliberately left behind. Other uncollected items: Health Power 1 at 18015 and 19063, Zapper at 23062, and Health Power 2 at 24589 and 27545. No new weapon-attachment pickup was reported missed. Ordinary cash audit: 61 likely collected, 46 missed, including the two complete boss batches within the collected count. This run does not prove optimal pickup collection.

Shop 1 upgraded Double Shot at 21405 (cash 2550 to 50). Shop 2 bought Drone at 29288 (5450 to 450). Full captured reports and gate/AVI evidence: work/l2-source-comparison/r277-shops.md, r277-pickups.json, r277-gate-proof.json, r277-avi-qa.json.