Autopilot · write-up

Level 4 native pilot validation, September 26–27, 2026

xenondoc/LEVEL4_NATIVE_VALIDATION.MD · 37 KB · updated 2026-10-01

The subsequent full-level attempt and separate second-half continuation are reported in LEVEL4_FULL_VALIDATION_20260927.MD. That attempt exposes a first-boss handover stall despite the earlier focused success, plus corridor/final-encounter projectile hits; do not treat the focused results below as a successful uninterrupted full-level validation.

Reproduction

Live control is the resident C pilot in Hatari. Python records and analyses the captures; it does not supply gameplay decisions. Build both consumers with:

powershell -ExecutionPolicy Bypass -File xenon_tools\hatari_dev.ps1 build-kernel
powershell -ExecutionPolicy Bypass -File xenon_tools\hatari_dev.ps1 build -BuildDirectory out\build\codex-validate

The full comparisons restore xenon_tools/run_logs/native-level3-visible-20260914/end.sav (frame 62579, 39 shields, one life), continue through the Level 3 shop into Level 4, and record both AVI streams with checkpoints every 150 frames. This historical snapshot already contains Rear Shot. The new Level 4 pickup policy refuses new Rear Cannon pickups, even if no Side Shot is installed; other levels retain their existing pickup policy.

$env:XAP_LEVEL4_TRACE_FILE = "E:\xenon_runs\level4-trace.log"
python xenon_tools\validate_campaign.py native-run level4-check `
  --output-root E:\xenon_runs `
  --resume xenon_tools\run_logs\native-level3-visible-20260914\end.sav `
  --build-directory out\build\codex-validate --frames 30000 `
  --stop-at-level 5 --checkpoint-interval 150

Hatari is visible. Stop recording gracefully through the run's stop-request file before terminating its process, so both AVIs receive their final indexes. The optional trace logs the mission goal, proposed input, planner input, final input, verification flag and predicted contact for each frame. Without the environment variable, no trace file is opened. The trace's applied movement precedes the session's fire pulse; use captured consumed input for the complete game input.

Missing models and conflicting steering

ReVa's MCP endpoint was unavailable. The local Level 4 dump xenon_tools/level_capture/level-04.stram.bin supplies the following evidence:

• $4F924 wraps shared sine-script movement ($E1E) and updates its animation. Capture previously omitted this wrapper's script bytes. It now captures the tape and uses the existing C script interpreter, including camera dependence. In run r18, the prediction at frame 65739 matched the body's subsequent X positions and its world Y after including the camera movement.
• $4F9CC drifts down one world pixel per frame and fires on an 8-count byte carry at object offset $5E. Left status $D8 fires direction 10 from (X+26,Y+26); the right side fires direction 13 from (X-26,Y+26). Speed is 4 in the game's direction table. Captured accumulator 242 at frame 65206 predicts allocation at 65208; the first actual projectile position agrees within one pixel. C now includes this future shot in the scene.
• $4F44A is an edge source that creates $4F61E minions and also fires. Both this source and $4F9CC use ordinary damage callback $E2A, with captured health 8 in these runs. They previously lacked the damage-supported scene flag, so combat evaluation could not credit destroying them.

The Level 4 source tactic selects a visible upstream source (screen Y from -16 to 144, bottom at least 32 pixels above the player's anchor), routes below it with an installed forward mount, and retains a target preference while it remains eligible. It does not chase a source that has already passed. Both physical route search and maneuver hazard evaluation remain active.

Two post-planning rules were also competing with validated movement:

• The lower-exit recovery could force DOWN while a curved enemy was sweeping past the bottom clamp. It now supplies a proposal before evaluation instead of replacing the result afterwards.
• Projectile escape could steer toward an incoming projectile. In r21 at 63351–63353, the planner chose UP-LEFT, but the final mask became RIGHT. At 66309–66310, a verified boss approach was similarly replaced by RIGHT. The duplicated projectile steering scan is now removed; the shared scene evaluates those projectiles when choosing each maneuver.

The separate projectile, sweeper and chain steering scans have now been removed. The prepared C scene already validates these hazards. A second scan can replace the nominal route before it is evaluated as well as overwrite the winner afterwards. Pickup steering overrides are also removed. The generic validated movement is authoritative, except for the tightly scoped entry camera egress described below. Camera progression at the lower clamp remains a proposal submitted to the evaluator.

The entry wall recovery remains scoped to its authored geometry: scroll 3800–4100, X 200–270 and world Y 4190–4232. It holds DOWN while leaving the right pocket so camera backtracking exposes the left opening, then crosses left until X <= 232. It prevents the previously observed wall stall. Its mission route is now limited to world Y 4140–4280 and scroll 3800–4100. The earlier broad left-side fallback replayed this old backtrack even after the ship passed the pocket, producing goals behind the ship.

Reachable wall shooters take priority over the upper entry crossing and may receive combat attack proposals. Shooters behind the ship's forward lane are not pursued. The old blanket exclusion of right-wall shooters from a left-side position is removed; reachability and bounded progress checks decide whether their mounted firing station can be used.

When no wall station is reachable, the entry dogleg aligns at X=80, climbs to the continuous open row at Y=4056, then crosses toward X=228. These short handoff stations use 12-frame alignment and 24-frame crossing forecasts. Diagnostic forecasts from r28 showed that a 56-frame rollout reaches the intermediate station, holds there and predicts later collisions instead of continuing the next leg. This fallback still requires full live validation: r29 reached the opening but took a projectile hit at frame 64439.

Level 4 span waypoints now choose a clear orthogonal dogleg when one exists. The mission checks horizontal-then-vertical and vertical-then-horizontal against the physical span graph, and both the nominal seed and continuation follow that axis order. horizontal_first retains its ABI field name: positive means horizontal first, negative means vertical first, zero allows a diagonal. Other levels keep their existing axis selection. The previous diagonal connector/ship-speed mismatch repeatedly pinned the hull where horizontal shots could hit it; r31 lost its life at 64797 after this sequence.

Recordings and intermediate results

All filenames below are under xenon_tools/run_logs/<name>.validation/; the recording is <name>.x2events, with <name>-original.avi and <name>-sprite.avi beside it.

Subsequent recordings use E:\xenon_runs\<name>.validation\ because D: is nearly full. The new native-run --output-root option retains the same artifact layout and writes the selected root into the manifest's command-line arguments.

Installed weapons and source attacks

The combat rollout previously spawned future centre and side fire, but not future wing Cannon fire. Captured Cannon bullets were already supported. C now reads the observation's installed Cannon offsets once when preparing combat and emits a round from each wing on a forward firing edge. The native combat scene and its ctypes inspection mirror both include these offsets. Hatari and the DLL must be rebuilt together after this ABI change.

The Level 4 dump's $4902 attachment and $4E50 projectile handlers confirm the wing X offset and upward speed of 10. Captures around r32 frame 64503 confirm the first collision centre at approximately ship Y minus 15. The model uses one round per forward firing edge; actual Cannon animation can fire more often (roughly every 4–5 frames here, versus an 8-frame centre reload). This is a conservative damage estimate. Future continuous Laser beams are not added by this change.

The optional attack selector previously accepted only swarms and wall shooters. It now also accepts the ordinary enemies in the scene's supported damage selection, which includes the reviewed Level 4 edge guns, spawners and sine enemies. It reuses the existing trajectory and chooses the nearest centre or installed Cannon firing lane. Each target contributes at most one station and the selector still proposes at most two attacks. No additional per-target path forecast is built. Every proposed attack is evaluated for collision safety.

In r34, 14 edge guns reached zero/negative captured health before destruction, versus 11 in the same frame interval of r33. This is a focused comparison, not a full-run improvement claim: restoring the checkpoint resets mission state, and the changed inputs also alter subsequent spawns. Both intervals had no shield loss. Positive-health cleanup transitions are excluded from those kill counts.

First-boss encounter retention

The old bypass criterion assumed fewer than five visible satellite heads meant one had been destroyed. Initially, some heads are simply above the view. That prematurely selected a firing route through the lower swinging head. In r33 at 66021–66025 the live trace already reported body contact and verified=0; these hits are not evidence of a false-clear prediction. The bypass now completes only after the ship physically passes the lowest live visible head.

After r35 cleared all five satellite heads, the main head remained alive below the view. Original handler $50554 checks the remaining-head count at $D82 before allowing damage to it. C now retains the nearby offscreen main head within 512 world pixels below the camera, returns down the exterior, and uses the right-facing Side Shot when installed. The main head was defeated in r37. R36 still took the two projectile hits listed above. The revised return station and curved-chain model were subsequently validated together in r59, below.

In r20, remaining shield loss was at 63355 (4, projectile), 64294 (8, scripted enemy #561867), 66133 (4, projectile), and first-boss projectile hits 66707, 66737, 66904, 66970, 66978, 67110, 67150, 67249, 67598. Frame 64421 restored shields to 39; its collision callback is a health pickup, not a shield-loss incident. The incident monitor now distinguishes this case.

Focused tests cover mirrored edge-shot directions and carry timing, capture of the new script wrapper and its camera dependence, damage-supported source selection, upstream firing stations, passed-source rejection, Rear Cannon exclusion, and health-recovery reporting. Level 4 is not yet validated as complete.

Moving enemies around the walls

The ordinary scripted enemies using $E1E/$E2A and compound pairs using $E1E/$E30 plus $E36/$E30 were omitted from supported combat targets. Source selection now recognizes captured script motion and decoded damageable health instead of repeating a per-level update-handler whitelist. This allows the existing attack proposals and combat rollouts to value destroying them.

For $E30 (thunk to $6160), object +$56 is the health owner. Damage is applied to that root, then the linked pair is destroyed together. Both collision shapes now share one combat target and death time. Missing or unselected owners remain hazards; an attachment's stale health word is never substituted.

Recognition alone did not prevent r40's rear-wave hit. The local second-bend tactic stays at X=96 while world Y is in (1824,2016], before the central passage opens at Y=1824. It allows the same supported attack proposals and leaves room beside the hull. Its same-start focused r41 validation passed without damage. The $4F44A minion spawner's direct shot still depends on uncaptured mutable overlay globals; that shot is not claimed to have an exact scheduled model.

Attack proposal selection also previously ranked only the nearest firing lanes. Known aimed periodic sources (captured spawn rate >= 100), reviewed edge guns and wall shooters now receive a 24-pixel ranking preference. The existing 48-pixel maximum detour, mounted lane constraints and collision evaluation remain unchanged. There are still only two attack proposals; no additional root prediction or unbounded enemy chase is introduced. Linked overlapping shapes with one health owner count as one projectile hit; different owners and unknown blockers still cannot receive ambiguous damage credit.

The fixed edge gun $4F9CC has an 8-count byte accumulator, so it fires every 32 frames. The native body scene previously included only its next allocation even when the horizon covered later carries. It now appends every deterministic allocation inside the requested horizon, moving each spawn down by the source's one-pixel world step per frame. This remains entirely inside C; the existing next-shot descriptor API and ctypes layout are unchanged. The regression fixture checks allocations at frames 1, 33 and 65 and their source displacement.

R53 exposed a separate combat-preparation rejection: the first boss's chain segment $5072C (#568217) was an unmodeled fire blocker, and a missing prediction for it rejected the entire lazy combat operation at 66340. Such unmodeled blockers now retain their observed hull, matching body preparation's stationary fallback. They never receive damage or kill credit through this fallback. A missing path for a credited target still fails preparation. The recorded failing frame now recomputes successfully with the current DLL. Optional decision-error lines in XAP_LEVEL4_TRACE_FILE identify the rejected preparation stage.

R51's fatal first-boss contact exposed a loadout difference: that historical run has Rear Shot and no Side Shot, whereas r35–r48's successful focused boss runs installed Side Shot. The first-boss corrections below address its forward-attack fallback and swinging-face prediction. The shooter changes alone did not establish a complete Level-4 pass.

The local dump explains why this encounter needs a separate motion model: $5060C chooses a player-dependent compass direction when the swing phase is zero, may choose a random angular speed, then advances a captured 0..24 swing phase. $5072C copies its predecessor's position and sine state through $506FA, then integrates through $F08/$9ACA. $E60/$3D58 only prepares its collision bounds. The face at $507F6 follows the main head at $50782. The former face forecast extrapolated the predecessor's live velocity for at most twelve frames; it did not reproduce this curved chain. The stationary fire-blocker fallback fixed rejection, not that collision model.

Ordinary navigation now retries its physical span graph when the graph enlarged for extra travel clearance has no route. Previously it stayed in a vertical fallback despite a hull-sized route around the wall. R51 passed r50's stall location with this fallback; actual movement still passes collision evaluation.

First-boss curved-chain prediction

xenon_autopilot_level4_swing.c now prepares both first-boss chains together inside the shared C prediction owner. Each child integrates its parent's same-frame sine state, including fixed-point fractions and wrapped 68000 arithmetic. The primary root stays fixed while the three segments and main head integrate; the face copies the main head instead of trailing it by a frame. The current primary bend is exact until extension reaches zero and the next update chooses a new player-dependent aim and possible random speed.

That unknown restart uses bounds over the five possible aim directions and 32 possible angular speeds. These bounds are generated once per world and reused, instead of treating an entire circle as occupied or guessing a random trajectory. The primary maneuver horizon includes the remaining deterministic bend plus eight reactive frames, with a minimum of 16 frames so observed projectiles retain an adequate escape window. There are fresh observations on every game frame.

The secondary root at $509DC has a deterministic 512-frame clock. C recovers it from the captured angular phase, delta and acceleration, then predicts the eight links at $50A50 and the secondary head at $50A60. Collision preparation uses the angle-dependent headers, a union of face orientations, and endpoint sweeps with a small two-pixel reserve. Paths and collision sweeps are prepared once for the whole chain, with no new Python gameplay bridge or ABI fields.

The earlier-start r60 test uncovered a separate integration bug: the resident driver's contact-kind whitelist omitted LEVEL4_BOSS_CHAIN and LEVEL4_BOSS_MAIN_HEAD. Their paths could be inspected in replay, but their collision bodies never entered the gameplay scene. Secondary segments at $50A50 consequently hit the ship at 67114–67118 even while the planner marked the movement verified. Both kinds are now included. A native driver regression asserts that the primary chain, secondary chain and main head reach its hazard selection; the test failed on the preceding DLL with all three absent.

Mission issues also affected safety and progress. Offscreen satellite heads were being counted as absent, prematurely ending the exterior bypass; the mission now retains all captured heads until destruction. A main-head attack without Side Shot now selects an installed forward firing station below the swing, rather than waiting at an exterior station that could not hit it. When Side Shot is available, the secondary head's firing height uses its predicted position at projectile arrival. Its avoidance horizon is 16 frames: eight was too short to escape a close directional projectile in r58. Fixed heads are cleared before targeting the secondary tip. Forward-only satellite stations have a shallow bottom limit instead of extending behind the primary chain; if a dodge takes the ship below it, the mission returns through the exterior. The handoff also accepts four-pixel route quantization: an authored Y=2494 landing at navigation Y=2496 must advance the phase, rather than hold forever because the exact Y threshold is not reached.

All following runs used visible resident C control, current Hatari/DLL builds, both AVIs and checkpoints every 150 frames. Their recording paths are E:/xenon_runs/<name>.validation/<name>.x2events:

R59 starts before any head is destroyed, with Side Shot installed. The four fixed satellites die by 67108, the secondary head by 67697, and the main head by 69965. The recording contains the zero-health destruction handlers for all six, rather than merely their disappearance from the viewport. R64 repeats that start after restoring the missing collision kinds and retaining the 16-frame escape window. Its fixed heads die at 66574, 66642, 66794 and 66989, the secondary head at 67482, and the main head at 70569.

The reusable validate_level4_swing_prediction.py checks C forecasts against future captured positions and verifies that the prepared collision sweeps cover the actual interaction hulls. For r64, stride 13 checks 319 forecasts, 53,186 exact positions and 150,674 hulls, with zero position error and zero uncovered pixels. Position checks stop at an unknown primary re-aim; hull checks continue through it. The comparison uses the update-pass camera origin (scroll_y + scroll_applied_delta), matching C observation decoding, rather than the earlier draw-pass origin.

python xenon_tools\validate_level4_swing_prediction.py E:\xenon_runs\level4-first-swing-20260927-r64.validation\level4-first-swing-20260927-r64.x2events --stride 13

Native tests cover the previously fatal curved pose, all same-frame links, restart uncertainty, shared preparation, offscreen bypass retention and the forward-only firing fallback. R64 establishes a clean focused first boss fight with its recorded Side Shot loadout and the complete collision model. It predates the radial-burst correction below. R66 also completes the earlier entry without damage after the radial-burst correction below. These focused results do not establish a damage-free complete Level-4 run.

Anticipated secondary radial burst

R65's projectile #571356 was one of eight rounds allocated at 66971 by the secondary head at $50A60, not a mispredicted chain segment. Its captured byte at +$5E was 249 at 66970, then carried after adding 10. The original read-only Level-4 descriptor at $4F07F/$4F080 supplies increment 10 and speed 6; the local dump and retained live RAM snapshots agree. Directions are compass indices 0–7. The burst spawned beside the ship, leaving only seven reactive frames before the up-left round hit it at the upper clamp.

C now retains that byte in the private shared swing capture and appends the next eight radial paths to the collision scene when the carry enters the horizon. Their anchor comes from the already-prepared exact secondary path. They remain independent hazards after their source dies. No new observation ABI or Python bridge is required. The subsequent random reload is not guessed; its next observation supplies the new captured timer.

A new bullet's update applies the camera step once more after allocation. Its future birth-frame value is unknown until the candidate executes, but is bounded by two vertical pixels; the scheduled bullet mask includes that reserve. The reusable checker additionally compares burst allocation, movement and hull coverage. On r65, stride 19 checks 3,506 bullet positions with maximum error two pixels and no uncovered collision hulls. The chain positions still match exactly. A regression checks all eight pending bullets and excludes a carry beyond the requested horizon.

Final focused validation

R66 restores r56's checkpoints/f66264-periodic.sav, before the first-boss approach, with 39 shields, one life and no Side Shot. It collects Side Shot during entry, destroys the four fixed heads at 66540, 66622, 66831 and 66908, the secondary head at 67737, and the main head at 71006. The shop opens at 71125. All 4,861 observations retain 39 shields and one life, with no recorded player hits. This proves the earlier-entry encounter with its naturally collected Side Shot; it is not a forward-only complete-fight result.

The stride-13 prediction check examines 364 forecasts, 72,062 exact chain positions and 183,500 collision hulls: position error and uncovered extent are both zero. It also checks 9,149 anticipated bullet positions, with at most the bounded two-pixel birth-camera difference and no uncovered bullet hulls. Both AVI RIFF lengths match their finalized file sizes.

python xenon_tools\validate_level4_swing_prediction.py E:\xenon_runs\level4-first-swing-20260927-r66.validation\level4-first-swing-20260927-r66.x2events --stride 13

Final-boss tongue prediction

The original Level-4 dump is xenon_tools/level_capture/level-04.stram.bin. $500D8 sets the tongue leader's clock-driven sine state through $509EA. $50126 followers and $50152 tail copy their predecessor's same-frame state through $506FA, then integrate sine substeps through $9ACA. $4FE74 advances the extension phase and firing byte accumulator; $4FE54 contains the vertical bob table. C previously treated these bodies as fitted linear motion.

The new xenon_autopilot_level4_tongue.c recovers the clock modulo 512 from the captured phase, phase delta and acceleration, testing only the two candidates implied by the acceleration wave. Prediction advances the captured firing accumulator and extension, then prepares the complete ordered chain once per horizon. In world coordinates the applied camera term cancels. Later unknown RNG reloads receive a conservative reachable envelope; they are not invented.

Collision preparation also models reactivation. A collapsed object's disabled collision X=1000 must not be translated into a future moving hull. Normal and tail sweeps use the reviewed collision headers and become empty after collapse. Existing prediction/scene inspection APIs expose the prepared data on demand; there are no new Python gameplay callbacks or per-frame telemetry requirements.

validate_level4_tongue_prediction.py compares the C paths with subsequent captured positions. On r43, 178 56-frame forecasts produced 159,488 position comparisons with maximum error one pixel and no errors above that. The r44 71730–71743 tail trajectory, including its launch, matches exactly and is a native regression test. Reproduce the broad capture comparison with:

python xenon_tools\validate_level4_tongue_prediction.py E:\xenon_runs\level4-final-region-20260927-r43.validation\level4-final-region-20260927-r43.x2events --stride 19 --horizon 56

The far eye is attacked from its own upper side pocket; the spent near eye can otherwise consume horizontal rounds. After both eyes die, the pilot stages above the lower sweep before aiming an installed wing mount into the main mouth. The mounted firing region is bounded vertically instead of extending all the way to the bottom clamp. The attack uses a 16-frame horizon and replans every frame so it can shoot during safe clock windows rather than require a fixed station to survive a whole 56-frame tongue swing. Movement remains subject to the same collision evaluator. R46 defeated this boss without shield loss; R48 confirmed that result after the station-state and waypoint corrections. The main attack is slower in R48 (shop 73460 versus 72697 from the same start), so this is a safety/progress result, not a boss-speed improvement claim.

R47 exposed a separate reached-waypoint deadlock: the ship at (62,41) satisfied the (60,40) goal band, but the next span segment's rounded start cell was blocked. The small quantization fallback summed X/Y offsets, rejecting 2+1 pixels even though both axes were within half of the 4px navigation grid. Level 4 now uses the maximum per-axis offset for this existing two-pixel fallback. Larger offsets still cannot skip a blocked corner, and the actual ship's movement still passes the collision evaluator. Other levels retain their previous tolerance rule.

R48's capture audit inferred collection of all 20 observed boss cash pickups; the game capture lacks explicit per-pickup collection events. Shop-entry cash was 6550. No equipment pickup appeared in this focused boss recording.

Final-head firing and cash follow-up (2026-09-28)

The resident C pilot already chose the forward Laser after the eyes died, but its 16-frame maneuver horizon often kept it away from the Laser's mouth lane. Keeping the same firing region and shortening this encounter's horizon to 12 frames let it reassess the clock-driven tongue more often. From the same level4-final-main-r105.validation/start.sav, the main head disappeared at frame 71285 in level4-final-forward-short-r3.x2events, versus frame 71676 in the earlier r105 recording. A second run from r105's frame-70932 checkpoint repeated the result at frame 71286 (level4-final-forward-short-r4.x2events). Both focused runs retained the earlier frame-70776 four-point hit where that frame was included; neither took new damage during the main-head phase.

The final boss already released its full 1,500 cash into the ship's path. In both follow-up recordings, cash rose from 5,500 to 7,000 before the shop opened, with no shop-entry cash adjustment. Eighteen drops were visible long enough for the pickup audit to classify beside-ship retirement; the remaining 150 cash was credited on the head's death frame. Unlike the first boss, this encounter needs no special cash routing.

The latest builds pass 96 targeted native tests covering prediction, resident driver selection, Level-4 missions, combat, source selection, world scenes, scheduled shots and maneuvers. Pickup auditing and shop reporting have separate focused tests from the preceding validation work. The complete mission test file also has four failing historical Level-2 fixtures; those are outside this Level-4 validation and are not claimed to pass. Earlier Level-4 entry shield losses still require a full same-start Level-4 validation. No complete-level clean-run claim follows from the focused second-bend and final-boss successes.