Autopilot · write-up

Autopilot performance: native/WASM kernel pilot

xenondoc/AUTOPILOT_PERFORMANCE_C_PORT_PLAN.MD · 154 KB · updated 2026-09-17

Hatari integration / protocol 16: the resident native session now runs directly from the emulator's canonical frame hook. Reusable C start/stop/status APIs own input; continuous driving has no Python/TCP/encoding/pause dependency. Recording and explicit debugging remain optional. The Python-hosted C session and Python reference path remain available. ABI stays 85; earlier sections below describe historical migration stages.

ABI 84 update: native driving and optional Python world views remove mandatory population reconstruction, with 35–47% lower recorded driving-processing time in the measured samples.

Current extension: native world preparation (2026-09-08)

AUTOPILOT_WORLD_PREPARATION.MD documents ABI 21: 54 scalar fields and base object classification now have a batch C decoder with common/per-level functions. Tracked objects store numeric kinds; legacy labels are compatibility views. Shared immutable enum groups and direct access to the native result buffer avoid new allocation/copying work. The Python backend stays available. Final tests: 941; recorded Level 1-5 verdicts unchanged. The controlled world-update ablation saves 3.49% observation time; C and refactored Python are approximately tied. This is not a complete old-executable performance comparison.

Lifecycle/draw aggregation, complex script states and legacy object construction remain Python work. The next boundary is native consumption of the typed decoded fields, followed by optional legacy views, rather than more micro-optimization of the scalar decoder. Historical sections below describe earlier migration stages.

Current extension: exact-motion consumers (2026-09-08)

AUTOPILOT_EXACT_MOTION_CONSUMERS.MD connects targeting and display to existing exact shell/script/formation paths, removes unused velocity requests and memoizes genuine fallback fits on demand. It removes 83.6% of measured fits; it does not demonstrate an overall speedup. Some replay decisions change, as documented in that report.

Current extension: native tracked velocity (2026-09-08)

AUTOPILOT_TRACKED_VELOCITY.MD adds ABI-20 velocity fitting over retained C history. Prediction and targeting reuse the result through the compatibility view without another history transfer or C call. This begins migrating consumers onto the native owner introduced in ABI 19. The fit is opt-in: measurements showed no net gain, and caller inspection found many shell/script consumers that should use existing exact paths instead.

Full-C direction and next work (2026-09-08)

The end goal is an autopilot that runs entirely in C. Python remains available for reference tests, recording inspection and replay observability; its diagnostic objects must not become a required runtime representation. A future HTML replay UI can consume the same exported diagnostic data.

AUTOPILOT_FIRING_LANE_REUSE.MD removes repeated free-map construction and firing-column scans before porting this search. It reduces total observation time by 4.9%; routing remains Python and still belongs on the full-C completion list despite its now-small runtime cost.

The first C-owned tracking slice is now implemented in AUTOPILOT_NATIVE_WORLD.MD, ABI 19. Lifecycle filtering, draw aggregation and handler-specific field decoding remain Python. C owns the canonical identity population, motion history and core geometry normalization.

Next migration work continues frame preparation beyond that initial slice. The latest profile attributes 8.398 cumulative seconds (4.678 self) to WorldState.update. Start with one batch of typed object/draw observations, identity/lifecycle tracking, coordinate normalization and motion history. Retain explicit update-pass versus draw-pass coordinates and slot-reuse semantics. Avoid rebuilding Python objects just to marshal the same state back into C; materialize Python diagnostic views only where an actual consumer needs them. During migration, Python tactics may still require a compatibility view; measure that cost separately rather than assuming the first port removes it all.

Then migrate remaining tactics, maneuver selection and persistent navigation/map ownership onto that shared C state. Keep transport outside the core API. Near the end, replace TCP serialization in the embedded build with direct in-memory frame input and action output; retain recording/debug adapters. Do not invest in a new C TCP/JSON stack solely for an interface the embedded autopilot will bypass.

Current extension: incremental terrain observations (2026-09-08)

AUTOPILOT_INCREMENTAL_TERRAIN.MD adds ABI-18 native tile projection/row comparison, unchanged-row reuse and batched cache invalidation. Historical cell bookkeeping and destructible-wall proofs remain.

Previous extension: combined native candidate trials (2026-09-08)

AUTOPILOT_COMBINED_CANDIDATE.MD adds ABI-17 combined forecasting/evaluation and shared-prefix accounting. Separate controls measure reusable buffers, packed actions and the combined C loop independently.

Previous extension: native candidate-scene preparation (2026-09-07)

AUTOPILOT_NATIVE_CANDIDATE_PREPARATION.MD adds ABI-16 batched pickup paths and reuses existing native shell buffers in candidate preparation. Separate old, shell-only and complete controls isolate their costs.

Previous extension: native combat-bound preparation (2026-09-07)

AUTOPILOT_NATIVE_COMBAT_PREPARATION.MD adds ABI-15 batched combat-bound preparation for supported stationary, linear and shared-path sources. Python retains unsupported geometry; source order and separate bullet-blocking envelopes are preserved. Existing-cache reuse removes zero-distance translations, without a separate stationary cache.

Previous extension: direct script bodies and dispatch bypass (2026-09-07)

AUTOPILOT_DIRECT_SCRIPT_BODIES.MD removes Python frame slices for generic script hulls already predicted in the shared scene, and bypasses script dispatch for non-script objects. Existing ABI-14 C collision and combat APIs perform the work; specialized models and old controls remain.

Previous extension: shared native prediction scene (2026-09-07)

AUTOPILOT_SHARED_PREDICTION_SCENE.MD moves captured script execution and ordinary formation delays to C. ABI 14 adds named script/formation descriptors; policy and body/combat consumers borrow an observation-owned point pool. Reference paths remain selectable, and the API is still callable from Python without Hatari driver integration.

Previous extension: native policy swarm contacts (2026-09-07)

AUTOPILOT_NATIVE_POLICY_PATHS.MD moves scripted swarm contact evaluation into a batched C query over shared path predictions. ABI 13 adds xap_first_path_contact; unsupported sources keep the Python predictor. The library remains callable from Python, without Hatari driver integration.

Previous extension: world draw preparation (2026-09-07)

AUTOPILOT_WORLD_DRAW_PREPARATION.MD removes geometry collection for draws without surviving tracked objects and uses a single render rectangle directly. Wall tiles and multi-draw object bounds retain their existing behavior. This reduces Python world preparation; native ABI 12 is unchanged.

Previous extension: batched host frame decoding (2026-09-07)

AUTOPILOT_BATCH_FRAME_DECODING.MD removes per-field Python reads from repeated fixed protocol records using standard-library native unpacking. This improves the Python driver and binary replay input path; it is not a new prediction kernel or a direct Hatari integration. ABI 12 is unchanged.

Previous extension: preparation reuse and deferred results (2026-09-07)

AUTOPILOT_PREPARATION_REUSE.MD describes shared observed maneuver setup and deferred Python motion objects. Additional combat and targeting caches were removed after showing no measured benefit. Native evaluation and ABI 12 remain unchanged. Reached prefixes still consume the same budgets; only the selected path is fully materialized for replay.

Previous extension: policy preparation and navigation grids (2026-09-07)

AUTOPILOT_POLICY_PREPARATION.MD documents local fallback wall probes, shared neutral swarm forecasts and the ABI-12 configuration-grid builder. C produces free cells and Manhattan clearance in one call using the resident observed wall raster. Python still owns route search and level tactics; the Python grid builder remains selectable. No Hatari driver integration or WASM profiling is included.

Previous extension: whole-candidate evaluation (2026-09-06)

AUTOPILOT_NATIVE_CANDIDATE_PLAN.MD describes the ABI-11 ordered evaluator, lazy combat, shell releases, pickups and ranking aggregates. C also advances periodic/composite shots and both homing families; Python supplies bounded escape/barrier geometry when needed. The previous ordered evaluator remains available as the c-scene benchmark control, and the complete Python kernel remains selectable.

Previous extension: remaining scene models (2026-09-06)

AUTOPILOT_NATIVE_SCENE_MODELS.MD documents ABI-10 source partitioning, fallback eye delay chains and deterministic next enemy shots. C prepares shared paths once per observation; the Python reference, unsupported shot models and level tactics remain available. Per-stage replay comparisons measure each change separately.

Previous extension: whole-maneuver forecast (2026-09-06)

AUTOPILOT_NATIVE_MANEUVER.MD documents the ABI-9 continuation, player/camera stepping, wall and independent-body validation loop. The Python driver calls it once per maneuver, including fresh candidate branches. Ordered dependent-hazard, combat and policy checks remain in Python; the complete Python reference stays selectable. No Hatari integration is added.

Previous extension: shared scene and terrain preparation (2026-09-06)

AUTOPILOT_NATIVE_PREPARATION.MD documents compact action keys, resident native wall bands, projected corridor intervals, batched shell paths and direct native slice preparation. These use ABI 8 and remain callable from Python, with the reference paths retained. No Hatari integration or WASM profiling is included.

Previous extension: combat rollout (2026-09-06)

AUTOPILOT_NATIVE_COMBAT_KERNEL.MD describes the ABI-6 ordinary-shot rollout and destruction-aware body checks, including simplified validation and removal of Python health/death synchronization. The standalone library remains callable from Python through ctypes. There is no new Hatari driver integration in this stage, and WASM is not profiled.

AUTOPILOT_NATIVE_LIVE_VALIDATION.MD records the committed ABI-6 Level 1 native/Python live comparison: identical gameplay, 15% lower mean gameplay decision time and 19% lower p95 with C. Its fresh profile recommends compact action-cache keys first, then batched native wall/corridor queries and broader native hazard-scene preparation. Their implementation and separate measurements are now recorded in the preparation document above.

Previous extension: linear prediction preparation (2026-09-06)

The next stage reuses constant-velocity parameters once per observation, then lets C prepare eligible future collision sweeps from compact integer/fixed-point descriptors. The shared native/WASM boundary now uses ABI 4, with a contact flag result and scene validation separated from repeated queries. See AUTOPILOT_NATIVE_LINEAR_KERNEL.MD for rounded velocity conversion, specialized-model fallbacks, diagnostics and measured stages.

Previous extension: batched body queries (2026-09-06)

The second port adds independent-body validation to --kernel-backend c, with matching C/Python numeric hazard types, originally with named-struct ABI 2. See AUTOPILOT_NATIVE_BODY_KERNEL.MD for the boundary, reference/combat fallback, validation and isolated performance comparison. Python remains selectable. Maps and level tactics remain in Python.

First port: articulated-eye forecast (completed, before body extension)

The first port is the complete articulated-eye trajectory forecast, not a per-frame ship-motion function, scalar rectangle predicate, navigation-grid builder, or legacy beam search. The same dependency-free C99 source now builds as a shared library driven by Python and as a WASM object/module. Python remains the default and reference; C is explicitly selectable with --kernel-backend c.

The original analysis below is retained as historical evidence. Its hotspot ordering and proposed ABI are superseded, following the maneuver planner, spans navigation and architecture optimizations. The current CLAUDE.md permits builds using hatari_dev.ps1; its old "manual build only" description was stale.

Fresh profiles and first-kernel selection

Both samples use the committed fcb070e0 architecture, all five features, spans navigation, the same warm-up/checkpoint/terrain sidecars and no live emulator work. cProfile identifies call structure; its instrumented times are not end-to-end speed estimates, and nested cumulative times cannot be added.

The eye chain makes 602,112 integration calls in 784 forecasts, including 9,301,424 sine lookups. This is the strongest isolated, numeric, stable kernel in these samples: one captured controller determines the entire linked chain, with no player dependence, terrain, shooting, RNG or policy callbacks.

The corridor's larger group-query cost remains a future target. It includes lazy heterogeneous motion prediction, bounds construction, cache/hash work and destruction-sensitive queries. Porting only its scalar distance function would leave most work in Python and introduce frequent boundary calls. A useful port there needs an owned scene/trajectory representation and batched validation. Ship/camera arithmetic alone is now too small to justify choosing it first.

Profiles: xenon_tools/run_logs/native-kernel-{corridor,boss}-0906-01.pstats and the matching per-frame timings. Timing comparisons below run without cProfile.

Implemented boundary and data ownership

src/autopilot/xenon_autopilot.{c,h} originally exported ABI version 1 (the current linear-body extension requires ABI 4; the eye struct layout is unchanged):

int32_t xap_abi_version(void);
int32_t xap_eye_chain(const XapEyeControllerState *state, int32_t depth, int32_t horizon,
                     int32_t *out_xy, int32_t capacity);

• One call per controller per observation runs all future frames, links and sine substeps. Input is a named XapEyeControllerState struct mirrored by Python's ctypes.Structure. Its eight 32-bit fields occupy 32 bytes; typical output is 8 links × 96 frames × two 32-bit coordinates (6,144 bytes), link-major.
• C owns the exact, immutable 256-byte sine table. It uses defined unsigned 32-bit wraparound, explicit signed high-word extraction and integer waves; no host sine implementation, signed-overflow dependency or negative shifts.
• Caller-owned buffers are not retained. Python converts output once to ordinary point tuples; no borrowed C buffers enter replay/checkpoint state. Calls are reentrant and do not require a mutable native context in this pilot.
• C compile-time assertions and Python tests verify the struct size and every field offset. The named struct preserves ABI version 1's binary layout; no packing pragmas or per-field foreign calls are needed. phase is explicitly unsigned in both languages; all other fields are signed.
• The public header specifies supported dimensions, state ranges, capacity and return codes. Unknown/unarmed controllers return no forecast, matching Python. Invalid arguments fail without writing output. Callers provide aligned, disjoint buffers as required by the C ABI; the Python wrapper validates ranges.
• Python retains controller association, tactics, maps, collision validation, combat and picking actions. Do not transfer the map to C yet: this kernel never reads it, while Python navigation and replay still do.

For a later terrain/validation kernel, C should retain packed immutable wall rows and masks, then accept observed row/tile patches. Reset/rebuild on namespace, level, incompatible mask or uncertain revision changes; replay restoration must not reuse stale mutable handles. Python remains the observed-terrain authority while it needs maps, and predicted kills must never open destructible walls. This is a future boundary design, not functionality claimed by the current pilot.

Selection, builds, recording and replay

./xenon_tools/hatari_dev.ps1 -Action build-kernel
python xenon_tools/autoplay.py --planner maneuver --navigation-backend spans --kernel-backend c
python xenon_tools/replay_ui.py RECORDING.x2events --kernel-backend python

build-kernel builds only the optimized shared library under out/build/autopilot-native; it does not stop or relink Hatari. Native Python uses ctypes, without CPython headers or an extension module. Set XENON_AUTOPILOT_LIBRARY for a different compiled library. An explicit C selection fails clearly on a missing library or wrong ABI; there is no silent fallback that could invalidate a benchmark. The default is python.

--kernel-backend python|c works with live autoplay, recording, campaign validation, offline profiling and replay. Explicit selection overrides restored checkpoint values. Recordings preserve the effective backend and library ABI/hash; campaign recordings archive the DLL and C sources. Replay shows recorded and analysis kernels separately, and preserves an override through reset/rewind.

The existing Hatari WASM CMake target links the same object library. The isolated module can also be built without SDL/emulator dependencies:

emcmake cmake -S src/autopilot -B out/build/autopilot-wasm -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build out/build/autopilot-wasm
node xenon_tools/check_native_eye_wasm.mjs out/build/autopilot-wasm/xenon_autopilot_wasm.js xenon_tools/run_logs/native-eye-vectors-0906-01.json

This exposes the ABI for browser reuse; it does not implement the remaining autopilot in JavaScript/WASM or connect a browser gameplay driver.

Verification and measurement

benchmark_native_eye.py compares all 512 wave phases, clamp/timer/cycle boundaries, long chains and captured boss controller states. Its generated vectors validate every output coordinate through the WASM ABI as well. Initial native and WASM checks match 654,496 points across 1,316 cases.

The 8-link/96-frame microbenchmark, including ctypes packing, allocation and conversion back to Python points, measured 3.997 ms Python vs 0.119 ms C. A separate runtime-only Python sine-table cache measured 3.080 ms. Including that control is essential: repeated sin() calls are avoidable Python work, but caching them still leaves most of the linked fixed-point integration cost. The checked-in Python oracle is unchanged. Microbenchmark results are not whole autopilot speedups. Node/WASM measured approximately 0.023 ms with input copying and output-array conversion; its representation/runtime differs from ctypes, and this is not a phone benchmark.

Use compare_kernels.py RECORDING --from-frame FIRST --to-frame LAST --repeats 2 --output NEW_DIRECTORY for fresh-process end-to-end comparisons of Python, table-cached Python and C. It holds planner settings/observations constant, reverses variant order, fingerprints sources/library/assets, and rejects changed actions, tactics, contact verdicts, predicted kills or pickup counts.

Final boss results (844 observations, two complete opposite-order passes):

C reduces whole-observation mean time by 29.7% versus existing Python, or 24.3% versus the table-cached Python control. Baseline means differ by only 0.04% between passes. All compared decisions/verdicts match in both passes. There are 784 native forecast calls across 844 observations, not one call per substep, member or candidate. Policy time falls from about 5.01 to 1.27 ms; maneuver validation time stays about 5.8 ms. p95 improves only 3.8%, identifying remaining slow frames outside this kernel. This pilot targets the boss and does not claim a comparable corridor/campaign speedup.

Boss comparison, fingerprints and raw timings. The vector/microbenchmark report is xenon_tools/run_logs/native-eye-vectors-0906-01.json.

Inactive encounters (two opposite-order passes, workers pinned to the same permitted logical CPU with --pin-cpu; normal gameplay is not pinned):

Both samples make zero native forecast calls and have zero compared decision differences. The small 0.4%/1.4% mean differences do not demonstrate a meaningful overhead or gain. The barrier's pre-existing unverified verdicts remain unchanged; this is not a new Level-5 gameplay validation.

The first unpinned corridor experiment is excluded from the overhead estimate: its Python means varied from 23.675 to 26.756 ms, while C varied from 23.107 to 18.225 ms, despite no C forecasts. This is timing drift, not evidence of a kernel speedup. All raw results remain available. Pinning is a benchmark-only control; the comparison tool now warns if any variant's repeated means differ by >15%.

• Corridor accepted timing
• Corridor excluded timing
• Level-5 timing

830 tests pass after the named-struct follow-up, including optional native tests (the library was built, so they ran rather than skipped), ABI capacity/range checks, controller boundaries, 32-bit wraparound, concurrent calls, independent output storage, checkpoint selection, missing-library failure, and replay override/rewind. A real Tk smoke run replayed boss frame 8050 with C, displayed the formation overlay and one native call, then reset to Python and reproduced the selected input. Native and WASM builds both passed the same trajectory vectors. WASM module size is 8,246 bytes in this standalone build, excluding its JavaScript loader.

No new live campaign or phone benchmark was run. The complete Hatari browser application was not rebuilt; the standalone WASM module and shared source were compiled/tested. The struct follow-up rebuilt both native and WASM targets and rechecked all 654,496 golden trajectory points on each. This completes the Python-driven feasibility pilot. Keep the Python oracle and explicit backend selection while extending the native boundary to any subsequent profiled kernel.

Historical analysis — pre-maneuver planner (superseded)

The following sections explain earlier fixes and discarded migration proposals. They do not define today's optimization priorities, ABI or acceptance criteria.

1. Read the profile before designing anything

From the user-supplied cProfile run (16 canonical frames, xenon_tools/run_logs/full-game-03.validation/full-game-03.validation/offline-timing.cprofile.pstats) and the checked tank201-offline-profile.pstats table already in PROFILING.MD:

is_navigation_anchor_free's own body is 5ms total — practically everything under it is _ensure_navigation_grid. Grid construction alone is ~41% of the whole 6.985s profiled interval, and the checked tank201 profile shows the same shape at larger scale (grid work = 7.63s of choose's 15.32s, PROFILING.MD lines 113-126, which already says: "investigate actual cache misses, revision changes, and repeated geometry variants before changing search width." This document does that investigation.

2. Why the grid rebuilds so often (the "don't call it" analysis)

Update, measured: the analysis below originally hypothesized three compounding causes, led by "revision churns constantly." That hypothesis was checked with direct instrumentation (diagnose_revision_churn.py and diagnose_grid_cache_keys.py, monkeypatching _observe/_ensure_navigation_grid at runtime, no source file modified) against the exact recording and frame range the profile came from (full-game-03.x2events, frames 25377-25392) and turned out to be mostly wrong. Kept struck through for the record — one part (windowing) is still the right fix, now for a precisely measured reason instead of a guess.

~~1. Cache scope is per-map-revision, and revision churns constantly... during any~~ ~~actively scrolling segment... the entire grid cache is wiped essentially every~~ ~~frame.~~ Measured: false for steady gameplay. PersistentWorldMap._seed_resident_level (world_map.py:262-312) installs the entire level's static tile map in one pass at level start (min_row=0, max_row=rows-1 immediately) — there is no incremental "discovery" of geography to churn on. After seeding, _observe's changed = prior is None or prior.solid != solid (world_map.py:357) can only still fire on a genuine live/resident disagreement (a destructible tile opening, or a recording starting mid-level with stale resident state). Instrumenting the exact profiled range showed revision was stable at 78 for the entire 16-frame window — 0 bumps — and across the whole 5404-frame warm-up from level start, only 77 bumps occurred total, clustered at two points (frame ~19989: 20 events, first observations of rows beyond the resident map's extent; frame ~23056: 57 events, a new-solid batch) — nowhere near the profiled window.

2. The rebuild scans the whole discovered map, not a local window — confirmed, and this is where essentially all the real cost is. _ensure_navigation_grid (world_map.py:1836-1930) builds a prefix-sum table over self.min_row..self.max_row — because of point 1, this is the entire level (measured: 301 rows, navigation_rows=1204 at 4px resolution) from very early on, not something that grows slowly over a run. Timing the individual rebuild calls in the profiled window directly: 5 genuine rebuilds occurred, costing 210-310ms each (1218ms total) out of the whole ~7s profiled interval. Every caller only ever queries a handful of anchors within a few dozen pixels of the ship's current position, but each rebuild pays for the whole level regardless.

~~3. Multiple call sites request different reserve values within one decision, each~~ ~~forcing its own rebuild.~~ Measured: mostly false. The cache key is (revision, *geometry, sprite_id) (world_map.py:1837). Instrumenting every call in the profiled window found only 3 distinct geometries ever requested (matching the three reserve tiers actually in use) and 1602 of 1607 calls were cache hits (99.7%) — 1416 of those via the fast _navigation_key == key shortcut, no dict lookup even needed. Multiple reserves per frame is real but the multi-slot cache (world_map.py:1986-1990, up to 32 entries) already absorbs it almost perfectly.

What was actually measured to cause the 5 rebuilds

The cache key also includes self._player_collision_sprite_id, which changes as the ship's collision sprite cycles through animation/banking frames. The profiled window saw 4 distinct sprite_id values combined with the 3 geometries — up to 12 working combinations contending for the 32-slot cache over the 2300+ frames since the last revision bump. The 5 rebuilds are cold-starts/evictions of specific (sprite_id, reserve) pairs — most visibly the rarely-needed reserve-0 "emergency" tier, evicted between uses and recomputed in full when a hazard next needs it. Real, but secondary — the big lever is on rebuild cost, not rebuild frequency, since frequency is already low (5 in 1607 calls).

What to fix before touching C

• Window the grid to a bounded region around the player (a screen or two of margin, sized to comfortably cover the beam search's horizon and the retained A route's lookahead) instead of self.min_row..self.max_row. Measured potential: a ~250ms average whole-level rebuild (1204 nav rows) would shrink to touching maybe 40-60 rows instead — roughly a 20-30x cut on the only genuinely expensive part of this path, and it doesn't touch the collision model at all. This is still the single biggest available win*, now precisely quantified instead of guessed at.
• Increasing the 32-slot cache cap is a legitimate, secondary, complementary lever — it would reduce how often the 5-ish rebuilds happen by giving more sprite_id x reserve combinations room to stay resident — but it does not fix what makes each rebuild expensive when it does happen, so it's worth far less than windowing on its own and is not recommended as the primary fix.
• Prefer is_player_mask_anchor_free over is_navigation_anchor_free at call sites that only need a single anchor's free/blocked answer, not a persistent grid for subsequent A*/route search. It already exists specifically to avoid the full rebuild (world_map.py:1109-1114 docstring says as much).

Expect windowing alone to remove most of the ~1.2s of genuine-rebuild cost in the profiled ~7s interval (~17% wall-clock; likely a larger share of the reported 2.109s cProfile self-time, since cProfile's per-call overhead scales with the huge whole-level inner-loop iteration count each rebuild performs). Re-profile after this change before deciding what, if anything, still needs C — 3a below may turn out to be unnecessary once rebuilds are this cheap.

Optional follow-on: a reserve-independent representation

Windowing shrinks the area a rebuild touches; it doesn't remove the fact that today each (sprite_id, reserve) pair gets its own full grid. A further, independent lever collapses the reserve dimension out of the cache key entirely, so it stacks with windowing rather than replacing it.

The reserve dilation in _player_navigation_mask (world_map.py:1000-1033) and the tile-fallback extents in _ensure_navigation_grid (world_map.py:1931-1946) are both symmetric dilations (square/rectangle structuring elements) of the base ship silhouette by reserve. Dilation/erosion duality means "ship dilated by R overlaps a wall" is exactly equivalent to "the undilated ship overlaps a wall raster dilated by R" — so the dilation can be pushed onto the wall side, which depends on neither reserve nor sprite_id:

1. Wall distance field D(x,y): distance to nearest solid pixel, same metric the existing dilation uses. One computation per revision/window, independent of ship or reserve.
2. Per-sprite ship-clearance field: Clearance(anchor) = min D(p) over pixels the base (reserve-0) ship silhouette covers at anchor. One erosion pass per (revision, sprite_id) — not per (revision, sprite_id, reserve) as today.
3. Any reserve becomes a threshold compare: free_at_reserve(anchor, R) = Clearance(anchor) > R. No rebuild, no convolution, O(1) per anchor.

This collapses the cache key from (revision, geometry, sprite_id) — measured as 3 geometries x 4 sprite_ids = 12 combinations contending for 32 slots — down to just (revision, sprite_id), 4 combinations, removing the reserve dimension as a source of eviction pressure entirely.

The existing clearance byte array (world_map.py:1948-1975) is already the same two-pass forward/backward distance-propagation sweep step 1 needs — it's just currently computed after a reserve-specific free grid, measuring distance-to- invalid within one committed reserve rather than distance-to-wall directly. Restructuring it to run once on raw wall occupancy, with one erosion pass per sprite added on top, evolves code that's already present rather than adding something foreign.

Sequencing: do this only after windowing lands and is re-profiled. Measured impact here is secondary (~3x further, on top of windowing's ~20-30x, and only for the sprite-churn-driven evictions specifically — 5 rebuilds out of 1607 calls in the sampled window). It's also materially more implementation work (a real distance transform, not just narrower loop bounds) touching a path both A* and the beam search depend on, so it needs the same differential-testing discipline as anything else here. Build it only if sprite-driven eviction still shows up as a real cost across a full campaign after windowing, not on the strength of this one sample.

Implemented: caller-aware tactical windowing

Landed in world_map.py/autoplay.py. Before implementing, re-reading find_navigation_path's docstring (world_map.py:1745-1756, unchanged by this work) surfaced a hard constraint the original §2 plan missed: strategic route search deliberately searches to the frontier of the entire discovered map, because a past windowed/truncated search picked a route into a dead end that only looked safe because the trap was outside the window. Windowing _ensure_navigation_grid uniformly would have silently reintroduced that exact bug class for route planning while fixing tactical cost. The fix is therefore caller-aware, not a blanket change:

• _ensure_navigation_grid takes an optional row_window (world_map.py:1868); None (the default, used by every existing caller unless explicitly opted in) preserves full-map behavior exactly. is_navigation_anchor_free takes a keyword-only tactical: bool = False (world_map.py:1088) that computes a quantized window via _tactical_navigation_row_window (world_map.py:1847) only when explicitly requested.
• Window size: TACTICAL_NAVIGATION_WINDOW_MARGIN_TILE_ROWS = 40 (640px each side) and ..._STEP_TILE_ROWS = 8 (128px quantization step, so the window — and the grid cache key — stays stable across many consecutive frames rather than shifting on every anchor). Margin is sized from the largest verified tactical horizon, LEVEL2_HOMING_ESCAPE_PLANNING_HORIZON = 80 frames at the ship's max vertical speed (400px), with comfortable slack even at the least-favourable snap phase.
• Marked tactical=True at exactly two call sites, both audited individually for "does anything downstream of this call need the full map": the beam search's grid-priming call in _formation_escape_decision (autoplay.py:14955, verified bounded by the same <=80-frame family of escape horizons) and _prepare_navigation_collision_grid (autoplay.py:24118, verified its only caller's lookahead loop is bounded by <=32-frame horizons). Every other caller — find_navigation_path, find_navigation_path_to_region, find_navigation_path_to_row, find_navigation_connector, navigation_corridor_center, and _ensure_navigation_route's reserve-fallback call — was left untouched and still builds the full-map grid.
• Two real bugs surfaced and were fixed during implementation, both in the windowed tile-level SAT fallback path (used when precise pixel mask data isn't available): an IndexError from a bounds check that still compared against the true map extent instead of the active window (world_map.py:1949 area), and a separate bounds check in the pixel-prefix branch that conflated the (now possibly smaller) windowed navigation-row count with the always-full-map pixel prefix's own height (world_map.py:1901-1908, 1965). Both were caught by the existing test suite, not by inspection.

Verification performed: - Full test_autoplay.py + test_level_tile_maps.py suite: 583 tests pass. Two pre-existing test_level2_strategy.py failures were confirmed unrelated by reproducing them identically against unmodified code (git stash) before this change — not a regression introduced here. - Differential decision capture (tactic, movement_authority, last_dx/dy, navigation_reserve per frame) run before/after via git stash, byte-identical on both: the original 16-frame profiled window (full-game-03.x2events, frames 25377-25392) and a full independent 300-frame Level 5 recording (level5-barrier-clearance-0830-118.x2events, different level, different hazard types, run start-to-finish). - Rebuild cost re-measured on the exact original profiled window: the 3 rebuilds now routed through the tactical window (geometry=(14,15,14,10), the beam search's emergency reserve) dropped from ~210ms to ~54ms each (navigation_rows 1204 -> 324) — roughly the predicted ~4x. The other 2 rebuilds (geometry=(26,27,26,22) and (22,23,22,18), both row_window=None) are confirmed still full-map, as designed — they come from _ensure_navigation_route's strategic reserve fallback (autoplay.py:8849), which correctly was not made tactical. Total measured rebuild cost in this window: 1217.8ms -> 764.8ms (~1.6x overall), smaller than the earlier ~20-30x estimate because that estimate assumed all 5 rebuilds were windowable; 2 of 5 are legitimately strategic and were correctly left alone. The remaining cost is now concentrated in strategic route rebuilds — a different, out-of-scope lever (see §2's "reserve-independent representation" follow-on, or reducing how often _ensure_navigation_route re-triggers a reserve-fallback search) if it's still worth pursuing after further profiling. - Re-ran the identical profile_autoplay.py replay ... --cprofile command against the same recording and frame range (full-game-03.x2events, 25377-25392) end to end. Direct before/after comparison:

_score staying flat is a useful sanity check: it doesn't call _ensure_navigation_grid directly, so it should be unaffected, and is. The 16.2% total-interval win, smaller than _ensure_navigation_grid's own 38%, is exactly explained by the 2-of-5 strategic rebuilds correctly left untouched above. Outputs: xenon_tools/run_logs/full-game-03.validation/full-game-03.validation/ offline-timing-after-windowing.{pstats,timings.jsonl,summary.json} (original offline-timing.* preserved alongside for comparison).

Implemented: full campaign validation and a second, much bigger fix

Ran a new full campaign per CAMPAIGN_VALIDATION.MD (validate_campaign.py run full-game-04-windowing-validation-0903-02 --connect --port 6903, attached to an already-open idle Hatari left paused by an earlier finalized run rather than fighting over the port). It played from assets/xenonplay.sav (frame 26175) through Level 1 to a boss encounter and stopped at the documented no_lives_remaining condition at frame 34321 -- a real stop condition, not a crash. 139 slowdown periods were recorded (slowdowns.jsonl).

validate_campaign.py slowdowns full-game-04-windowing-validation-0903-02 (per PROFILING.MD "Find slow live frames cheaply") surfaced slowdown #31 as the standout: frames 27948-28029, 23 slow decisions, peak 705.4ms, tactic level1_left_corridor. Reconstructed offline (PROFILING.MD "Reconstruct offline with per-frame timings") using the nearest earlier checkpoint's sidecar -- the slowdown record itself names the correct one (checkpoint_before), which is always the checkpoint immediately preceding the slowdown's onset, exactly the "immediately before this recording" contract --controller-state requires: checkpoints/L1-f27897-periodic.sav.autoplay.json. Profiled with --cprofile (slowdown31-profile.pstats, 82 measured frames, 140M function calls, 37.995s instrumented).

Is it required? Is it called too many times?

_score's own body was the new #1 self-time consumer (4.839s tottime, up from 0.242s in the §1 sample -- this scenario is a heavy swarm_escape/survival:space_time_beam encounter, not the light §1 sample). Underneath it, target_priorities -> _weapon_reaches -> _weapon_direction -> _shot_path_clear (autoplay.py:22574- 22994) was a distinct, previously-invisible cluster costing 4.7-4.8s cumulative each, unrelated to the navigation grid entirely.

_shot_path_clear (autoplay.py:22842) checks whether a firing ray to a candidate target is blocked by wall geometry. Its last check, for the persistent (not live-draw) wall map:

for cell in self.world_map.solid_cells:      # rebuilds a fresh list over EVERY
    if cell.row == target_row and ...: continue   # discovered cell on each access
    cell_rect = Rect(cell.world_x, cell.world_y, TILE_SIZE, TILE_SIZE)
    if ray.intersects(cell_rect):
        return False

ray is a strip a few pixels wide between the ship and the target. solid_cells (world_map.py:1047-1048) is a property that reconstructs [cell for cell in self.cells.values() if cell.solid] -- for a resident-seeded level, thousands of cells -- on every single access, and this loop then tests every one of them against a tiny local rectangle. 2444 calls in this window x up to ~6000 cells each is the direct arithmetic explanation for the measured cost. This is not called "too many times" in the sense of redundant caching (unlike the grid) -- it's each individual call doing thousands of times more comparison work than it needs to, the same root cause (unbounded scan for a local query) as §2's finding, in a completely different subsystem.

How to reduce the cost

The file already has the right pattern one function up: _world_walls_near (autoplay.py:23260-23280) bounds a query to the tile bins the query rectangle actually overlaps. Applied the same idea: compute the ray's own tile-row/column range and look cells up directly from self.world_map.cells (already a dict[(column, row), MapCell]) instead of scanning the rebuilt solid_cells list. Implemented in autoplay.py:_shot_path_clear. No behavior change -- same cells get tested, just located by direct lookup over the ray's own footprint (typically a handful of tiles) instead of a full-level scan.

Verification: full test suite (583 tests) passes. Differential decision capture (same fields as §2) over the full 82-frame slowdown-31 window, before/after via git stash: byte-identical. Re-profiled the identical window:

The weapon-targeting cluster dropped out of the profile's top 20 entirely. The ~10.6% total-interval win (smaller than the ~4.7s absolute savings might suggest) is because this cluster, while individually enormous, was one contributor among several very large ones in this specific heavy-combat sample (_ensure_navigation_grid is still 5.363s tottime here, _rect_clearance 2.969s cumtime -- see below). Outputs: xenon_tools/run_logs/full-game-04-windowing-validation-0903-02.validation/ slowdown31-profile{,-after-shotpath}.{pstats,timings.jsonl,summary.json}.

Is it suitable for C translation?

The fix here was a Python-level bug (unbounded scan), not an algorithmic cost that needs a faster language -- exactly the "code that doesn't run is the fastest code" case, same as §2. Once bounded to the ray's own footprint, _shot_path_clear's remaining cost (0.068s cumulative for 2444 calls) is no longer a meaningful target for a C port on its own. If it becomes hot again in a different scenario, the bounded tile-lookup loop is small and POD-friendly enough to fold into the same C navigation kernel already proposed in §3a, but there is no evidence that's needed yet.

What's left in this sample, and what it means for §3

Two costs remain large in slowdown-31's profile and were not touched by either fix so far, because they are algorithmic cost, not wasted work:

• _ensure_navigation_grid (5.363s tottime here). This is a heavier scenario than §2's sample (5400 rebuild-checking calls vs. 1607), consistent with §2's own finding that the windowed fix only removes some of the cost (the tactical share); the strategic-route share should still be present and may dominate more here given how combat-heavy this window is. Confirms §2's own recommendation to keep measuring rather than assume the fix generalizes uniformly across scenarios.
• _rect_clearance (2,283,044 calls, 2.969s cumtime, autoplay.py:25409). A tiny two-line Euclidean edge-to-edge distance function, called from deep inside the beam search's per-node clearance computation (autoplay.py:15590-15627): for every beam node explored, min() over _rect_clearance(ship, hazard) across every tracked hazard (swarm members, shots, homing states). Unlike _shot_path_clear, this is not over-scanning -- it's O(beam_nodes x hazards) by the search's actual design, and a swarm/combat scenario genuinely has many of both. Required: yes, each beam node's survival scoring needs its true clearance to every live hazard. Called too many times: the call count is a direct, non-redundant product of the search width and hazard count, not duplicated work -- no cheap Python-side reduction was found here (one minor, low-value tweak: the min([...]) at autoplay.py:15590 builds a full list before taking the min; passing the generator directly to min() would skip that list-allocation overhead, but wouldn't touch the 2.3M _rect_clearance calls themselves). How to reduce further: would mean narrowing the search (candidate count or hazard set), which trades away the hazard horizon PROFILING.MD's gate explicitly forbids trading away for performance -- not pursued. C suitability: this is exactly what §3b already proposed (xenon_expand_candidates) -- a tiny, POD-in/POD-out, extremely hot inner loop (Rect x Rect -> float) called millions of times inside a search that's already algorithmically necessary. This sample is direct evidence for that plan, not a new finding: the fix here is porting the whole beam expansion to C, not a Python-side change.

One more: the same solid_cells waste, found by asking "don't we already track this?"

After the _shot_path_clear fix, grepping every remaining caller of PersistentWorldMap.solid_cells found the property is otherwise only used by the Tk debug UI and the offline route plotter (both fine with an occasional rebuild) -- except one: autoplay.py's live --trace writer (used by every campaign recording, autoplay.py:27222, main()) wrote "map_solid_cells": len(world_map.solid_cells) into the per-frame JSON trace record, every canonical frame of every live run. This is invisible to all the offline profile_autoplay.py replay profiling done above -- that path exercises ReplayModel.seek, not main()'s live trace writer -- so it would never show up in any of the .pstats files, only as live-run overhead outside decision_ms (which PROFILING.MD's "three different costs" table already notes excludes "UI/AVI, JSON serialization").

The map's own solid/free state is already tracked incrementally and up to date -- self.cells (a plain dict) is updated cell-by-cell in _observe and in bulk once in _seed_resident_level; nothing about solid_cells was a cache of that, it was a full rebuild-into-a-list on every access regardless. Since the only remaining production caller needed just a count, added PersistentWorldMap._solid_cell_count, maintained as an O(1) delta exactly where _observe's existing changed = prior is None or prior.solid != solid already detects a transition, recomputed once in _seed_resident_level (cheap, runs once per level) and reset in clear(). Exposed as solid_cell_count; autoplay.py:27222 now reads that instead of len(solid_cells). solid_cells itself is untouched (still correct for its existing UI/offline callers) but now documents the cost at the definition (world_map.py) so a future hot-path caller doesn't repeat the mistake.

Verification: full test suite passes. Wrote a standalone check (asserts solid_cell_count == len(solid_cells) every frame) and ran it across the entire new full-campaign recording end to end -- 8,147 frames checked, 0 mismatches -- including through the Level 1 boss encounter (destructible/solid transitions included, not just the static resident seed).

3. What's left that's genuinely CPU-bound (candidates for C)

Two clusters survive scrutiny as real, portable, numeric hotspots — as opposed to policy/orchestration logic that happens to show up in a flat profile because of what it calls:

3a. Navigation grid construction kernel

Even windowed, the per-cell prefix-sum (world_map.py:1862-1876) and per-anchor mask convolution (world_map.py:1877-1930) are tight nested loops over plain integers — exactly the "many small geometry calls" PROFILING.MD line 23 already flags cProfile overhead for. This is a clean POD-in/POD-out kernel: tile solidity in, ship-mask rows in, a flat free byte grid out.

3b. Ship/camera physics + candidate/beam expansion

game_model.py is already exactly what the migration strategy asks for: frozen dataclasses of plain ints (ShipState, CameraState, ShipTransition, CameraTransition), pure functions, zero dependencies (game_model.py:1-10 states this independence is deliberate). _advance_model_candidate (autoplay.py:12184-12215) is a thin wrapper called 63,937 times in this profiled interval alone — that's the beam/space-time search in _formation_escape_decision expanding one game-frame of physics per call, per candidate branch. This is the textbook case for the "batch the inner loop into one C call" guidance: today it's 63,937 individual Python-level calls; it should be one call that runs the whole branch-and-frame expansion natively and returns terminal states.

What NOT to port yet

_formation_escape_decision and _score show up big in the profile (4.757s / 1.645s cumulative), but their own bodies are hazard-classification and tuning-heuristic logic — dict/set membership over TrackedObject/WorldState, tactic-specific score bonuses, dozens of named constants (autoplay.py:12385-12444, 24145-24234 and beyond). This is precisely the "application/orchestration/policy logic" the user's own preferred strategy says to keep in Python — it's also the part still actively being tuned (per AUTOPILOT.MD, Level 4 heuristics are still moving). Porting it to C now would freeze the part of the codebase that most needs to stay fast to iterate on, in exchange for shaving time that's already mostly attributable to the two hotspots above, not to this function's own logic. Re-profile after 3a/3b land — if _formation_escape_decision's self time is still large once its children are fast, that's the point to look again, not before.

4. Proposed C ABI

Following the user's stated ABI conventions (fixed-layout POD only, coarse-grained batched calls).

// xenon_native.h

#include <stdint.h>

typedef struct {
    int32_t screen_x;
    int32_t screen_y;
    int32_t steering;
    int32_t speed_level;
    int32_t default_scroll_intent;
    int32_t scroll_y;
    int32_t scroll_backward_limit;
} XenonShipState;

typedef struct {
    int32_t scroll_y;
    int32_t scroll_intent;
    int32_t default_intent;
    int32_t override_age;
    int32_t forward_limit;
    int32_t backward_limit;
    int32_t backtrack_window;
    int32_t input_mask;
} XenonCameraState;

typedef struct {
    int32_t screen_x;
    int32_t screen_y;
    int32_t world_x;
    int32_t world_y;
    int32_t steering;
    int32_t scroll_y;
    int32_t scroll_intent;
    int32_t scroll_applied_intent;
    int32_t scroll_override_age;
    int32_t scroll_backward_limit;
} XenonShipCameraTransition;

// 1:1 native mirror of game_model.advance_ship_and_camera, for differential
// testing against the existing Python function before anything batched is built.
void xenon_advance_ship_and_camera(
    const XenonShipState *ship,
    const XenonCameraState *camera,
    int32_t input_mask,
    XenonShipCameraTransition *out_transition);

// One beam/candidate branch: a starting state plus the input mask sequence to
// apply, one entry per simulated frame.
typedef struct {
    XenonShipState ship;
    XenonCameraState camera;
    const uint8_t *input_masks;   // length = frame_count
    int32_t frame_count;
} XenonCandidateBranch;

// Batches the whole beam expansion that today drives 63,937 separate
// _advance_model_candidate calls into one native call. Caller supplies one
// branch per candidate; xenon_expand_candidates walks each branch's full
// frame_count internally and writes only the terminal transition (plus a
// per-frame collision flag array the caller already has to check against the
// navigation grid/masks) back out.
int32_t xenon_expand_candidates(
    const XenonCandidateBranch *branches,
    int32_t branch_count,
    XenonShipCameraTransition *out_terminal_states,   // one per branch
    uint8_t *out_frame_collision_flags,                // flattened, branch-major
    int32_t flags_capacity);

// Navigation grid construction kernel (windowed). tile_solid is a flat
// row-major array over the caller-chosen window, not the whole discovered map.
typedef struct {
    int32_t x_origin;   // ship-mask row's X offset
    uint32_t bits;       // packed occupancy bits for this row, matching the
                          // existing Python `expanded_rows`/`world_rows` shift-OR
                          // representation
} XenonMaskRow;

int32_t xenon_build_navigation_grid(
    const uint8_t *tile_solid,      // window_tile_rows * MAP_COLUMNS
    int32_t window_tile_row_origin,
    int32_t window_tile_rows,
    const XenonMaskRow *ship_mask_rows,
    int32_t ship_mask_row_count,
    int32_t ship_mask_x_origin,
    uint8_t *out_free,               // NAV_COLUMNS * window_nav_rows, caller-owned
    int32_t out_free_capacity);

Notes: - xenon_advance_ship_and_camera exists purely as the differential-testing anchor — it should never itself be the hot path; xenon_expand_candidates is. - xenon_expand_candidates takes the whole candidate/frame tree in one call, per the "coarse-grained" rule — this directly eliminates the 63,937-call count, not just makes each call cheaper. - xenon_build_navigation_grid intentionally takes a caller-chosen window rather than the whole map, so the Python-side windowing fix from §2 and the native port are the same change, not two. - None of these take Python objects, dicts, dataclasses, or TrackedObject/ WorldState references — only flat arrays and the POD structs above, matching the required boundary discipline.

5. Migration plan

1. Land the Python-side windowing/cache-consolidation fix (§2) first, with no C involved. Re-run the exact same profiled interval before/after (PROFILING.MD's "Optimization gate" already specifies this protocol: same revision, same recording interval, compare actions/targets/route validity, not just milliseconds). This alone may remove enough of the cost that the C work in §3 becomes optional rather than required — measure before committing to it.
2. Port game_model.py to C almost verbatim (xenon_advance_ship_and_camera), compiled both as a native DLL (ctypes on desktop, no extra dependency) and into the existing Hatari Emscripten/WASM target. Keep game_model.py's Python functions in place and add a score_candidates_native-style twin per the user's differential-testing pattern: python def advance_ship_and_camera_python(...): ... def advance_ship_and_camera_native(...): ... # ctypes call Run recorded .x2events states through both and assert identical results before anything downstream switches over — validate_scripted_motion.py and validate_game_model.py already establish this exact pattern for a different subsystem and can be mirrored directly.
3. Batch the beam/candidate expansion into xenon_expand_candidates. This requires reading _formation_escape_decision's branch-generation loop in more depth than this pass covered (only its entry/setup, autoplay.py:12355-12444, was read) to pin down exactly how branches are generated and pruned before finalizing the call shape above — treat the struct in §4 as a starting proposal, not final.
4. Port the windowed navigation-grid kernel (xenon_build_navigation_grid) once §2's windowing is in place in Python, so the window-sizing logic only has to be figured out once.
5. Re-profile after each step, not just at the end — §2 may already dominate the win, and porting 3a/3b when they're no longer hot would be wasted effort matching the "faster code is code that doesn't run" framing this task started from.
6. Do not port _score or the hazard-classification body of _formation_escape_decision as part of this pass — see §3's "What NOT to port yet."

6. Verification

Research/analysis only — no source was modified. Before implementing anything from this plan, re-run the offline profiling commands in PROFILING.MD §2-3 against the same recording/frame interval to get a fresh baseline, since this analysis is based on one already-captured profile.

World ownership follow-up (ABI 22)

C now owns scalar fields as well as tracks, with direct shell/pickup path preparation and optional Python field export. The current Python-driven planner still materializes compatibility objects. Controlled measurements show no speedup (~1.45% slower mean, up to 2% repeat spread); treat this as migration infrastructure. See AUTOPILOT_WORLD_PREPARATION.MD for API lifetimes, ablations and validation.

Script preparation and links (ABI 23)

C-owned captures now feed the existing script and ordinary-formation kernels without Python descriptor/byte repacking. Native updates export decoded script state rather than decoding it again in Python; policy links use C-resolved indices. The final controlled comparison is 3.30483 -> 3.28771 ms per observation (about 0.52%, a small change), with unchanged recorded Level 1-5 decisions. Complex model assembly, semantic grouping and tactical policy remain Python dependencies. See AUTOPILOT_WORLD_PREPARATION.MD for API lifetime and validation details.

Observation topology and static models (ABI 24)

Formation risk groups, ordered composite chains/member ordinals and common prediction model selection now live in xenon_autopilot_world_models.c. Python scene preparation consumes enum-selected sources and skips repeated eligibility scans; the script interpreter is unchanged. Reference paths remain. The controlled Level-1 mean is 3.32516 -> 3.33932 ms (0.43% slower, below the largest 1.61% repeat spread): this is migration progress, not a measured speedup. Grouping-only costs 0.31%; model selection/source-list reuse adds 0.12% relative to grouping-only. Recorded Level 1-5 decisions remain unchanged.

Next remove the Python consumers and exports together: ordinary follower-chain assembly/cache traversal and specialized model preparation still use Python link maps and object views. Let those native consumers read world indices and model records directly. Tactical policy and complete object materialization remain later migration work; TCP replacement is still a final integration step. See AUTOPILOT_WORLD_PREPARATION.MD for lifetimes, exact scope and validation.

Ordinary formation consumers (ABI 25)

After committing ABI 24 as e14d2111, the next consumers moved in order: (1) on-demand ordinary-chain traversal/cached-predecessor lookup, then (2) native whole-path bounds and sampled swept relevance. Root callbacks still use Python for specialized policy. Shared native point storage and the existing formation execution kernel are reused; preparation stays on demand.

Two pinned opposite-order Level-1 repeats measured 3.32741 -> 3.31056 ms overall (0.51% faster; up to 0.74% repeat spread). Chain traversal alone is 0.16% slower; relevance is 0.66% faster relative to that intermediate stage. This is a small migration result, not a large demonstrated speedup. Native/planner tests and recorded Level 1-5 decisions pass. See AUTOPILOT_WORLD_PREPARATION.MD for scope, controls, API lifetimes and artifacts.

Next: specialized composite/follow-target preparation, followed by its tactical consumers. Keep link-map removal tied to migrating their last Python users; porting another producer alone cannot remove compatibility export costs.

Linked and composite consumers (ABI 26)

The previous phase was committed as 3b6dce51. Same-frame follow/attachment paths, coupled composite trajectories and collapsed extension bounds now have C-owned-state preparation APIs and an optional Python bridge. Root policy stays Python where needed. Composite batches cache every member at once, eliminating the reference routine's repeated full-chain computation. Scripts and composites share one internal sine substep; no duplicate interpreter was introduced.

On the targeted Level-3 composite capture, two pinned opposite-order repeats measured 15.40394 -> 10.61094 ms (31.12% faster), with unchanged decisions. The separate stages were 2.13% slower for linked-only paths, 27.01% faster for composite batching relative to that, and another 7.60% faster for collapsed bounds. Level-1 overhead is 3.32293 -> 3.34482 ms (0.66% slower; up to 0.70% repeat spread). Do not extrapolate the composite gain to all encounters.

96 native tests, 67 planner tests and 575 forced-native gameplay-policy tests passed. Standard Level 1-5 recorded comparisons and the targeted composite capture have unchanged decisions. A later composite recording hits an existing empty-candidate error in the retained baseline and could not be compared. No live campaign or Hatari runtime integration was performed.

Six compatibility link maps still have Python users. Next migrate camera-path dependency analysis and tactical link queries, then let native collision consumers read the already-prepared paths/bounds directly. See AUTOPILOT_WORLD_PREPARATION.MD for exact scope, artifact paths, API lifetimes and the baseline failure.

Direct linked collision consumption (ABI 27)

The previous phase was committed as cf35b8ad. Native linked points and collapsed bounds now feed requested collision slices directly through xap_pack_linked_slices. Python retains only source metadata, buffer ownership and ordered slot markers. This removes per-source/per-frame rectangle creation, union, numeric validation and ctypes repacking for eligible geometry. Animated attachments and specialized hulls retain authoritative dispatch; collapsed bounds already contain their guaranteed emitter sweep, including its frame-zero rule.

The 480-frame composite encounter improved 10.58268 -> 8.87599 ms (16.13%) in two pinned opposite-order repeats. Repeat spread stayed below 0.6%. Level 1 remained flat: 3.30816 -> 3.30517 ms, a 0.09% change versus up to 0.61% spread. All recorded Level 1-5 and targeted composite decisions are unchanged. Standalone build, 98 native tests, 67 planner tests and 575 forced-native gameplay tests passed. No live campaign, Hatari integration or WASM profiling was performed.

The six link-map exports remain. Next migrate camera-dependency analysis and remaining tactical link queries so their compatibility views can eventually be removed. See AUTOPILOT_WORLD_PREPARATION.MD for eligibility, buffer lifetimes, frame laziness, exact measurements and validation artifact paths.

Native camera and relationship flags (ABI 28)

C now resolves camera dependence once per observation over semantic follow-target links, reusing world scratch and the existing screen-relative handler classifier. It also computes the shared no-follow-target/no-parent/no-composite-membership eligibility flag. Broad phase, swarm threat, delayed-eye preparation and ordinary root checks consume the flag; the maneuver adapter consumes native camera flags. Synthetic/reference observations retain the previous Python checks.

Standalone build, 100 native tests, 67 planner tests and 575 forced-native gameplay policy tests passed. Standard Level 1-5 windows plus the targeted composite capture have unchanged decisions. Two pinned opposite-order repeats measured Level 1 at 3.29108 -> 3.26953 ms (0.65% faster) and the composite encounter at 8.80631 -> 8.94153 ms (1.54% slower). This is migration progress with mixed timing, not a broad performance win. Flag dictionaries and Python dispatch still cost work.

No link-map exports were removed: specialized/reference prediction and tactical queries still use all six. Next migrate those consumers together with their compatibility exports rather than adding another parallel representation. See AUTOPILOT_WORLD_PREPARATION.MD for semantics, controls and validation artifacts.

Native fallback-eye and Level-4 head preparation (ABI 29)

C now resolves fallback eye chains and prepares their root/delayed paths directly from owned tracks, scripts and links. It fits only independent fallback roots. Level-4 head paths preserve the separate live-velocity rule and twelve-frame cap. The Python bridge writes into the shared native scene pool, so anchor, displacement, collision-step and eye body/combat consumers reuse one lazy observation batch. The old assembler remains selectable with c-boss-paths-old.

Build, 105 native tests, 67 planner tests and 575 forced-native gameplay tests passed. Standard Level 2-5 and both targeted boss recordings retain their decision verdicts. Two pinned opposite-order repeats measured Level-1 boss at 2.31055 -> 2.31831 ms (0.34% slower) and Level-4 boss at 3.25004 -> 3.18055 ms (2.14% faster).

Eye/boss predecessor exports remain for unsupported/reference consumers. Next migrate articulated-eye controller discovery/member assembly (eyes.py) and its remaining compatibility boundary; its numeric forecast kernel is already C. This phase migrates the separate fallback delayed chains, not that controller assembly. No Hatari integration, new live campaign or WASM profiling was performed. See AUTOPILOT_WORLD_PREPARATION.MD for API semantics, lifetimes and artifacts.

The standard Level-1 window also retained all verdicts, with pinned means 3.27434 -> 3.32545 ms (1.56% slower). New-path repeats spread by 2.58%; the other boss results remain local gains/near-flat results, not a broad speedup. Full artifacts are linked in AUTOPILOT_WORLD_PREPARATION.MD.

Native articulated-eye assembly (ABI 30)

C now captures articulated controller state and resolves member successors in the owned observation pass. One batch assembles chains and writes the existing numeric forecast into the shared scene pool. Python returns NativePath views, removing per-frame tuple creation and later scene copies. The previous integer-output API and Python assembler remain available.

Controls separate the work: c-eye-assembly-old disables new eye capture and uses the previous assembler; c-eye-assembly uses C assembly but retains tuple results; c also enables direct shared output. Build and 771 correctness tests passed. On the targeted boss recording, two pinned opposite-order repeats measured 2.25802 -> 2.26477 -> 2.22455 ms. Assembly alone was 0.30% slower; buffer sharing improved the intermediate result by 1.78%, giving a combined 1.48% gain with all recorded verdicts unchanged.

Controller snapshots remain Python-visible because boss tactics and candidate preparation still read them. Migrate those consumers before removing that export. No Hatari integration, new live campaign or WASM profiling was performed. See AUTOPILOT_WORLD_PREPARATION.MD for API behavior, controls and artifacts.

The standard Level-1 comparison also preserved all verdicts: 3.27704 -> 3.27808 -> 3.24867 ms across the same three controls. Assembly alone was essentially neutral; shared output contributed a 0.90% improvement over it, for a combined 0.87% gain. This is a modest migration gain, principally from avoiding Python geometry materialization, rather than a significant numerical-kernel speedup.

Standard Level 2-5 recorded verdicts also remained unchanged; their one-repeat reports are run_logs/articulated-level{2,3,4,5}-0908/comparison.json.

Native boss-core tactical inputs (ABI 31)

C selects the active Level-1 core and prepares its target top/aim X once per observation. The adapter consumes that selection; native candidate evaluation reads a borrowed XapBossCore through the request instead of receiving its top from Python. The NULL-pointer scalar path remains available for reference cases. Non-core candidate requests no longer calculate unused controller geometry.

Python controller snapshots are now decoded lazily from retained immutable bytes on the native world path. This preserves reference/inspection access and old snapshots without eagerly constructing the full controller state each observation. The bridge guards borrowed native geometry by owner stamp and target identity.

Build and 776 correctness tests passed. The controlled boss comparison retained all verdicts and measured 2.24238 -> 2.23282 ms (0.43% faster). Treat this as a small migration gain. c-boss-core-old disables selection and lazy decoding for comparison. Full tactical state machines and predecessor-map consumers remain Python work. See AUTOPILOT_WORLD_PREPARATION.MD for API lifetimes and validation artifacts.

The standard Level-1 means were 3.29341 -> 3.29369 ms, effectively unchanged (0.008% difference). All verdicts matched in both repeats; standard Level 2-5 correctness comparisons also passed unchanged. Reports are run_logs/boss-core-level1-timing-0908/comparison.json and run_logs/boss-core-level{2,3,4,5}-0908/comparison.json. This is principally a migration step, not a broad performance improvement.

Decision-scene step 1: shared fire descriptors (ABI 32)

Candidate shell-release and anticipated-shot metadata plus player fire masks now use one C preparation call. Immutable C velocity tables replace repeated Python point arrays; 64-bit descriptors shrink from 56/152 to 16/32 bytes. Python still selects sources and predicts spawn anchors. The reference preparation remains available as c-scene-fire-old.

Build and 780 tests passed. Recorded decisions match on standard Levels 1-5 and targeted boss/composite windows. Controlled mean timings are essentially flat: boss 2.27095 -> 2.28168 ms, composite 8.89180 -> 8.88051 ms. Standard Level 1 was 3.35951 -> 3.28710 ms, but the reference repeat spread was 4.14%, so this does not establish a reliable speedup. This closes another preparation boundary and removes per-source allocations. See AUTOPILOT_WORLD_PREPARATION.MD for artifacts, API lifetime details and limitations.

Next: remove transient Python CombatRollout construction when preparing native combat. Native world fields already contain much of its health/projectile data. Validate, document and commit each bounded step before beginning the next.

Decision-scene step 2: direct combat initialization (ABI 33)

Native candidate workspaces now read captured player bullets and target health from the native world, avoiding an intermediate Python CombatRollout, Bullet objects and health/animation containers. Empty target sets skip initialization. Numeric combat flags are captured during object decoding; buffers remain caller-owned. The generic Python trajectory path and reference initializer stay available. c-combat-initial-old provides independent comparison.

Build and 784 tests passed. Two opposite-order pinned Level-1 repeats preserved all verdicts and measured 3.28617 -> 3.27309 ms (0.40% faster), a small improvement. See AUTOPILOT_WORLD_PREPARATION.MD for ownership and validation details. Next remove Python per-frame range construction when native combat geometry is already complete, and avoid allocating a blocker buffer for empty scenes.

Decision-scene step 3: combat range batching (ABI 34)

Fully native combat geometry now supplies frame ranges in one C batch. Empty scenes skip that call and the unused 256-bound allocation. Mixed scenes retain reference preparation. First-use workspace restoration no longer copies state back over the identical freshly initialized buffers. Ablation: c-combat-ranges-old.

Build, 788 tests and recorded decision comparisons across all five levels passed. Level-1 timing means were 3.32855 -> 3.28356 ms, but repeat spread exceeded the apparent gain; no reliable speedup claim. The confirmed change removes Python frame metadata and empty allocations. Full details and artifacts are in AUTOPILOT_WORLD_PREPARATION.MD. Next batch: homing state initialization and immutable homing animation tables, retaining source/launcher selection in Python.

Decision-scene step 4: homing initialization (ABI 35)

Observed Level-2/5 homing state and launcher-spawn construction now run in C, without temporary Python simulation states. C owns immutable Level-2 animation tables; Python retains source selection and predicted launcher anchors. Empty sources skip preparation, and launcher seeds omit unused captured motion fields. The retained control is c-homing-initial-old.

Build and 791 tests passed. All standard Level 1-5 recorded verdicts match. Pinned Level-2 means were 6.02185 -> 6.03082 ms, effectively flat; this is another native preparation boundary, not a measurable speedup. See AUTOPILOT_WORLD_PREPARATION.MD for table ownership and validation artifacts. Next remove repeated pickup-list scans and combine pickup descriptors with the existing native pickup path call; tactical pickup-exclusion policy stays Python.

Decision-scene step 5: combined pickup preparation (ABI 36)

Candidate construction reads the pickup list once. C fills descriptors alongside paths, reusing the already computed compass decision; empty sets skip the native call. Python retains tactical incompatibility policy. c-pickup-scene-old preserves the repeated scans and Python metadata for independent comparison.

Build and 794 tests passed; all standard Level 1-5 verdicts match. Pinned Level-1 means were 3.25001 -> 3.26129 ms (0.35% slower, effectively flat). No substantial runtime gain is established by these preparation steps; their benefits are less temporary work and more native responsibility. Details and artifacts are in AUTOPILOT_WORLD_PREPARATION.MD.

The remaining substantial scene boundary is source selection and memory ownership. Python still orchestrates scenes, predicts launcher/shot anchors, supplies tactical exclusions and handles unsupported reference geometry. Broader planner, mission, navigation and runtime orchestration also remain; the autopilot is not yet fully C-driven. Keep tests/reference/replay observability in Python and continue to validate, document and commit each migration separately. No Hatari integration yet.

Decision-scene step 6: native source routing (ABI 37)

Scored-source routing and next launcher/periodic/barrier/composite allocations now read the native world directly in one C pass. Python populates existing consumer lists once, retains risk grouping and falls back for unavailable canonical state. Periodic firing fields moved into both scalar decoders, removing their inline WorldState.update block. Native source flags and kinds are numeric.

Build and 803 native/planner/gameplay/decoder checks passed. All five standard level comparisons retained their verdicts. Pinned Level-1 means were 3.25354 -> 3.26289 ms, effectively flat within repeat variation. c-source-selection-old isolates routing; both variants share the scalar decoder cleanup. Full ownership, coverage and artifact details are in AUTOPILOT_WORLD_PREPARATION.MD.

Source routing is now native, but the consumer containers and scene lifetime are still Python-owned. Further gains require removing these bridge/orchestration costs rather than expecting each small arithmetic port to speed up the whole driver.

The follow-up instrumented profile identifies body-slice horizon preparation as more worthwhile than further scalar metadata wrappers: 56,000 frame_slices calls for 1001 observations, mostly empty or already native linked sources. Batch these frames next, preserving mixed-scene fallback and source ordering. Profile artifact: run_logs/source-selection-profile-0908/c-1.pstats. Its instrumented times must not be compared to the ordinary timing means.

Decision-scene step 7: batched body-slice horizons (ABI 38)

Empty and fully native linked scenes now bypass Python per-frame slice preparation. C packs remaining linked frames in one call, sharing its sweep helper with the retained single-frame API. Empty scenes avoid both the 256-slice buffer and empty linked wrapper. Mixed scenes keep their original geometry preparation. The common range helper is now xap_prepare_slice_ranges. c-body-slice-batches-old isolates this change while preserving the original control's frame assignments.

Build and 806 checks passed, and all five standard recorded-level verdicts match. Pinned opposite-order Level-1 means improved 3.23509 -> 3.16617 ms (2.13%). The separate diagnostic profile confirms frame_slices calls fell 56,000 -> 4,630 (91.7%), with unchanged logical geometry counts. This is a measured waste-removal win, unlike the largely neutral smaller preparation ports. Details, repeat spreads and artifacts are in AUTOPILOT_WORLD_PREPARATION.MD.

The native library is ABI 38 and remains callable from Python. Initial scored population selection, consumer-container ownership, risk grouping, specialized fallbacks and higher-level mission/navigation/planner/runtime orchestration still need migration before fully C-driven autoplay. No Hatari integration or WASM profiling was performed. Keep subsequent migration steps independently validated, documented and committed; do not add the individual benchmark percentages as if they were a measured cumulative result.

Scene ownership: fixed slice storage (ABI 39)

C now owns the remaining body-slice allocation, sized once from the observation's source counts and horizon. Ordinary bodies/wall shooters need at most one slice per frame; Level-5 tile-pair volleys need eight. Empty scenes allocate nothing. There is no native relocation, prefix copying or buffer clearing, and shared path geometry stays borrowed. Python views retain the C allocation's lifetime. Repeated maneuver header construction is also removed; mutable slice fields are refreshed.

This deliberately replaces a proposed growth/retired-buffer design with a bounded allocation. It does not assume that visible objects alone bound future shots. The reference c-scene-storage-old control remains available. Full consumer assembly, frame-range ownership and reuse across observations are subsequent migration work; this is the first ownership step, not a fully native scene.

Build and 803 checks passed (143 native, 67 planner, 18 shell/eye and 575 forced native gameplay-policy). Opposite-order CPU-pinned Level-1 means were 3.17767 -> 3.17110 ms, a 0.21% difference within repeat variation (0.48%/0.30%). Treat this as timing-neutral. No Hatari integration or WASM profiling was performed. See AUTOPILOT_WORLD_PREPARATION.MD for replay artifacts and remaining scope.

Scene ownership: native candidate outputs and header (ABI 40)

C owns generated candidate output buffers and their reusable evaluation header. Existing C preparation writes directly into those outputs; shared prediction geometry remains borrowed. Python no longer constructs the full candidate header for every trial, and lazy combat attachment is preserved. Fixed source counts and the horizon determine capacities, with no growth or relocation.

The c-candidate-scene-old control retains Python allocation/packing. See AUTOPILOT_WORLD_PREPARATION.MD for ownership contracts and validation results. Remaining ownership work includes homing initial states, combat buffers and maneuver frame ranges; cross-observation reuse and higher-level orchestration still need migration. The standalone native library remains Python-callable.

Build and 807 checks passed. Opposite-order CPU-pinned Level-1 means were 3.18877 -> 3.17579 ms (0.41% faster), a small improvement rather than a substantial speedup. Detailed repeat variation and replay/profile artifacts are recorded in AUTOPILOT_WORLD_PREPARATION.MD. Tests and reference controls remain in Python.

All five recorded-level verdicts matched. The short diagnostic profile reduced maneuver-header packing calls 4,938 -> 1,000 and combat-header packing calls 2,771 -> 94. Python view construction largely offset that saving, leaving total instrumented time essentially flat. The next ownership work should connect C preparation and evaluation directly and remove Python views/assembly where they have no remaining consumer, rather than add more small allocation wrappers.

Scene ownership: direct preparation and evaluation (ABI 41)

The native candidate preparation API now invokes pickup/fire preparation directly into owned outputs, constructs entry rectangles, and attaches borrowed geometry. The driver passes the returned scene handle directly to evaluation. It no longer creates candidate output-array views or assembles the generated header in Python. Known input counts replace len() consumers of generated outputs.

native_direct_scene.py keeps the direct path separate from compatibility/reference preparation. DirectCandidateScene.inspect() is explicitly requested observability; normal driving never invokes it. A real candidate-evaluation test forbids inspection and the old output-view constructors. The existing replay UI still uses recorded diagnostics; no new implicit per-frame export was added. Future UI adapters can request the native inspection view while retaining the scene owner.

The direct handoff retains deferred combat, immutable shared paths, fixed capacity and Python-callable native APIs. It also avoids constructing empty input arrays. Compatibility controls are c-direct-scene-old and c-direct-empty-inputs-old. Remaining Python homing, maneuver and combat inputs are still required functional consumers, not optional observability, and remain future migration work. No Hatari integration or WASM profiling was performed. See AUTOPILOT_WORLD_PREPARATION.MD for detailed coverage, tests and measurements.

Build and 810 checks passed. All five recorded-level verdicts match and every candidate scene in the tested windows uses direct preparation. The diagnostic profile removes 6,000 output-array view constructions, 1,000 generated-header views and 4,938 old packing calls; inspection is never invoked. Ordinary timing is inconclusive: the final mean is about 0.46% slower with about 3% repeat variation. No reliable overall speed claim is made. Detailed measurements and separate empty-input control results are in AUTOPILOT_WORLD_PREPARATION.MD.

Candidate workspace ownership and reset (ABI 42)

C now owns reusable candidate homing/sample/flag buffers and lazy combat snapshots, live targets/bullets and scratch. Reset and evaluation are one native operation. Target selection and capture remain deferred until combat is needed, and snapshots contain only the initialized bullet prefix. The evaluator's existing flag clearing is reused; no duplicate flag clearing or scratch clearing was introduced.

The direct path no longer constructs Python CandidateWorkspace/DependentSamples, NativeCombatRollout temporary arrays, bytes snapshots or per-trial homing/combat memmoves. Planner-facing membership and result reads remain functional consumers. The c-workspace-old control retains the reference implementation. Initial homing preparation, barrier compatibility paths and high-level planner orchestration remain migration work. Standalone C is still Python-callable, with no Hatari integration or WASM profiling. See AUTOPILOT_WORLD_PREPARATION.MD for validation and timing.

Final validation: 815 checks passed; all five recorded-level windows match. The controlled Level-1 mean improved 3.20754 -> 3.12448 ms (2.59%), with both orderings improving. Diagnostic profiling removes 4,938 Python homing-reset calls and the Python workspace/combat-initialization allocation bridge. Health capture precedes geometry preparation, preserving the reference's 94 combat scenes in the short profile. Detailed spreads and artifacts are in AUTOPILOT_WORLD_PREPARATION.MD.

Seeded native workspace (ABI 43)

xap_workspace_create_seeded initializes homing states directly in the workspace's immutable reset storage. The direct Python scene keeps only input seeds and identity keys; it no longer allocates initialized output states or sends them back for C to copy. Reference evaluation materializes those states lazily. The existing initialized- state C API and Python reference controls remain available.

Validation: standalone kernel build; 156 native tests and 67 planner tests passed. The Level-2 homing replay window (32577–33537) matched the retained workspace path. One timing repeat is a correctness check, not a reliable speedup measurement. The driving test forbids Python homing output materialization with a nonempty model.

This is the first bounded change in complete native scene preparation. Source selection and homing observation seed assembly still use Python. Remaining major stages: finish scene preparation; planner orchestration; missions/tactics; persistent map ownership; controller lifecycle; transport-free native integration and validation. Generate immutable map data from the JSON asset at development time rather than adding a runtime JSON dependency to C. Keep diagnostics and reference tests optional.

World-backed homing inputs (ABI 44)

The existing native observation capture now retains the homing animation/counter fields needed by candidate preparation. xap_workspace_create_world consumes ordered XapWorldHomingRequest records and initializes its reset snapshot directly. Python passes source indices, target indices and future launcher anchors; it no longer packs observed homing bounds, positions, ages, directions, animation or sprite fields. Empty selections avoid building a record-index lookup. The seed/state APIs remain reference paths. xap_workspace_initial_homing is an explicit read-only inspection API, never called for driving. c-world-homing-old isolates this migration.

Validation: standalone build; 158 native and 67 planner tests passed. Tests cover both homing models, short captures, launcher allocations, source ordering and invalid indices. Level-2 (32577–33537) and Level-5 (91740–97391) replay verdicts matched. Level-2 opposite-order repeats averaged 5.859 ms reference and 5.899 ms native; this 0.7% difference is inside reference repeat variation (3.7%), so there is no credible end-to-end speedup claim. This change removes a Python preparation dependency. Remaining homing Python work: source-list selection, identity keys and future launcher anchor requests. Broader scene, planner, tactics and runtime migration remain open.

Embedded resident level assets (ABI 45)

generate_native_level_maps.py converts the reviewed assets/xenon_level_maps.json into checked-in xenon_autopilot_map_data.c. The standalone kernel embeds all five maps (30,000 uint16 tile words) and all 32 destructible-group descriptors, including partial Level-3 clear regions and Level-5 emitter cells. Metadata uses named structs and enum kinds; diagnostic IDs are retained only for tooling compatibility. Unknown source fields fail generation rather than being silently discarded. --check verifies freshness without rewriting files; generation is independent of checkout line endings. Normal C compilation needs neither Python nor the JSON file.

xap_level_asset returns process-lifetime immutable data with no allocation or file I/O. Native live runs and controller-owned maps select embedded assets. The existing Python map currently requests a cached compatibility export at initialization; this export is not a requirement of the C API. Python reference/custom-map tooling keeps its JSON path. Mutable map state, pixel-raster preparation and map policy ownership still need migration. The C data contains original tile codes, not simplified solid rectangles; preserve pixel masks and the existing boss-arena scratch exception when moving those consumers.

Validation: standalone build; 162 native tests, 67 planner tests, 575 forced-native policy tests and 15 authored-map tests passed. Tests compare every tile and every group attribute, then seed all five levels with gates closed/open against the reference map. They also forbid runtime JSON loading on the native asset path. No per-frame speedup is claimed for replacing initialization-time asset loading.

C-owned wall raster assembly (ABI 46)

xap_tile_raster_create assembles 16-pixel tile patterns directly into the immutable five-word-per-row wall map used by native navigation and candidate evaluation. Inputs are compact tile pattern indices plus deduplicated masks; C consumes them immediately. The raster and its words use one allocation. NativeRasterRows supplies lazy reference row access; normal native navigation borrows the C header and performs no Python row export or upload. Borrowed headers retain the raster owner across map revisions. Missing/unknown tiles remain solid; transparent pixels and source offsets are preserved. The prior Python assembly is retained under c-raster-old.

Validation: standalone build; 165 native and 67 planner tests passed. All five standard replay windows matched. Tests also compare individual raster bits, forbid Python row materialization during native anchor queries and verify old-header lifetime after cache replacement. Level-1 opposite-order repeats averaged 3.129 ms reference vs 3.138 ms native (0.28% slower, inside the 0.68% reference repeat spread); no end-to-end speedup claim. The reference uploaded 192,000 bytes per full-level raster. Native mode eliminates that upload and the eager Python tuple of 4,800 large row integers. Geometry preparation is infrequent in this replay, so the ownership/memory improvement has little timing impact.

Remaining map work includes native mutable tile observations/destruction policy and mask-asset preparation. Python still selects the tile pattern inputs; this is not yet a fully native persistent map or controller.

Remove optional per-cell diagnostic copies during driving

Native live/controller-owned maps now use track_observation_details=False. Unchanged cells retain their immutable geometry objects instead of allocating fresh MapCell instances for timestamps and observation counts. Changed rows also reuse cells when geometry and destruction evidence are unchanged. Missing-observation counters and live-solid evidence remain active: two complete absences are still required to clear a destructible cell. Snapshot geometry remains immutable. Uncollected detail fields are explicitly None and map export includes observation_details: false; replay/reference tools retain full details by default. This does not remove collision or destruction checks.

Validation: 165 native tests, 67 planner tests, 575 native-world policy tests and six terrain-observation tests passed. All five standard replay windows matched. Level-1 opposite-order means: 3.14352/3.14260 ms with details, 3.01620/3.01407 ms without; mean improvement 4.07%. Other windows are single-repeat correctness checks. The c-map-details-old control retains details; C comparison workers otherwise model native driving's disabled details, while ordinary replay UI defaults remain unchanged.

This removes Python allocation waste before migrating mutable map updates themselves. It does not complete the remaining scene/planner/mission/controller migration.

Native mutable map observations (ABI 47)

XapMapState owns gameplay cell state and renderer-row history. One xap_map_state_observe call projects the visible band and updates monotonic static walls, clipped-row handling and two-complete-observation destruction evidence. Stable rows allocate nothing and export no cell data. Python receives only changed gameplay cells while its remaining navigation/tactic consumers still need a compatibility map. Optional per-cell timestamps/counts retain the reference path when requested.

xap_map_state_create_level initializes directly from embedded tiles/group enums, including the Level-2 arena scratch exception. Custom maps and restored Python checkpoints import their existing gameplay cells once. Native intervals are fixed; the Python adapter reserves border rows and reconstructs from current cells if a custom observation exceeds that interval. Explicit map edits, RAM synchronization, level changes and reference-mode updates discard the native owner. Checkpoint copies do not share mutable native history; they restore from copied immutable cells. Authored destructible membership is established before observation ownership begins.

Validation: standalone build, 169 native tests, 67 planner tests, 575 native-world policy tests and six terrain-observation tests passed. Randomized clipping/range/reset comparisons and partial-destruction checkpoint restores matched. All five standard replay windows matched. Level-1 opposite-order means: 3.00797/2.97977 ms reference and 2.96179/2.95857 ms native (1.13% improvement; reference spread 0.95%, so treat the timing benefit as modest). Native mode issued one state batch per frame and exported only 1.687 changed cells per frame on average.

Remaining map consumers include Python tile-mask selection and span-route search. Scene assembly, planner orchestration, missions/tactics and the native controller/ transport boundary remain migration work. No Python-free autopilot entry point is claimed by this stage.

Native corridor-span graph and search (ABI 48)

xenon_autopilot_routes.c now builds compact row spans and executes the complete bounded route search in C. Overlap queries replace stored adjacency lists, and a fixed decrease-key heap avoids duplicate queue entries. Graphs reuse search/output storage. The Python bridge caches graphs on the persistent map and materializes only the winning path. Per-decision navigation wrappers must not own this cache: an initial implementation did so and rebuilt nearly every search. Checkpoint cache dictionaries are separate; shared graph searches are serialized through result export because their scratch storage is mutable.

Validation: standalone build; 173 native, 67 planner and six terrain tests passed. Randomized tests compare 400 routes and expanded counts against the retained Python implementation. All five standard replay windows matched. Level-1 opposite-order means were 2.90180/2.99540 ms reference and 2.88376/2.90029 ms native (1.92% mean improvement, less than the reference repeat spread; no strong speedup claim). The persistent cache builds 0.01475 graphs/frame for 0.13014 searches/frame, versus 0.12221 builds/frame with the incorrect per-decision cache lifetime. c-span-search-old retains the Python graph/search for comparison.

Remaining work still includes scene assembly, planner search orchestration, missions/tactics, mask asset selection and a complete native controller entry point.

Maintain destructible-group openness during map updates (ABI 49)

Gate tactics previously allocated a list by scanning every map cell for each destructible_group_is_open query. C now counts known and solid members once at map creation and adjusts totals in the existing changed-cell update. Queries are constant time; group IDs are numeric in C and mapped once from diagnostic names by the Python compatibility owner. Unknown/empty groups remain closed. The reference scan remains available with c-map-group-scan-old. Custom/checkpoint map creation rebuilds totals from seeded cells; embedded levels need no external group arrays.

Opposite-order Level-3 means: 7.11200/7.15657 ms reference vs 6.89333/6.95961 ms native (2.91% improvement). Level-5 window: 2.02029/2.03088 ms reference vs 0.87666/0.87050 ms native (56.87% improvement). Both windows matched decisions in both orders. This is a window-specific benefit from repeated gate queries, not a claim of 57% overall gameplay improvement. Artifacts: group-count-level3 and group-count-level5 under xenon_tools/run_logs.

Tests cover all five embedded levels, randomized observation equivalence, two-observation clearing, reappearance, checkpoint restoration and a native-query test that forbids Python full-map iteration. Remaining high-level consumers and the Python-free controller still require migration.

Stream Level-5 barrier volleys inside candidate evaluation (ABI 50)

xenon_autopilot_barriers.c handles directional barriers and radial tile-pair volleys in the ordered native frame loop. Directional headings turn once per reached frame until allocation; the projectile moves on that same allocation frame. Compact observed seeds are installed once per scene. Trial headings use bounded stack storage (64 sources; the reviewed map has 17 controllers), and no per-frame projectile rectangles are exported to Python. Source order and the reference's contact-only barrier checks are preserved. Python preparation remains available through c-barrier-samples-old and unsupported custom inputs.

Barrier scenes now use the reusable native workspace. This removes the provisional evaluation, Python sweep preparation, mutable-state restoration and second evaluation previously required for every candidate in these scenes. The native loop still stops at the first ordered contact. Unknown-start wall handling retains its separate retry.

The active 300-observation barrier window (level5-barrier-clearance-0830-118.x2events, 90160–90459) matched in both timing orders. Mean processing time fell from 7.65648 to 5.24748 ms (31.46%). Candidate calls fell from 64 to 32 per observation; 32 barrier rechecks and 147.94 Python dependent checks per observation disappeared. The standard Level-5 tank window also matched but had no barrier sources and no measurable benefit. Reports: barrier-native-active and barrier-native-level5.

Validation includes randomized contact comparisons over both volley families, all eight headings, several carry delays, moving ship paths, repeated trials, atomic invalid-input rejection and a driving test forbidding Python dependent preparation. 177 native, 67 planner and 575 native-world policy tests passed; all five standard replay windows matched. Full scene assembly and the high-level planner/controller remain Python migration work.

Prepare barrier seeds directly from captured world fields (ABI 51)

xap_candidate_scene_set_world_barriers reads selected native tracks and decoded field-presence flags. It derives carry delays/headings and writes directly into final scene storage, omitting unsupported captures and volleys beyond the horizon. Python passes object indices and receives only source-slot mappings for contact identities; it no longer repacks barrier anchors, headings, family enums and fire delays during normal driving. The compact-seed API remains usable for custom callers and the c-barrier-seeds-old timing control. The shared byte-carry helper also serves source selection, avoiding duplicated accumulator semantics.

179 native and 67 planner tests passed, including direct-vs-reference seed comparison, missing-field/out-of-horizon filtering, source-order mapping and stale/invalid selection handling. The active barrier window matched in both timing orders for all three paths: old Python samples, C evaluation with Python seeds, and direct world preparation. Direct preparation measured about 0.75% faster than Python seeds in this short window; this is a small ownership improvement, not another large performance claim. The main gain remains removal of the second candidate evaluation. See world-barrier-active.

The barrier forecast/evaluation path is now callable entirely within C. Selecting the overall scene, body/combat assembly, missions, planner orchestration and driver lifecycle still contain Python dependencies.

Native deterministic wall-shot descriptors (ABI 52)

xap_world_prepare_scheduled_shots derives Level-3 small-wall-shooter and Level-4 directional-wall-shooter allocation timing, spawn positions, hull offsets and velocity from captured native fields. Common observed status is retained in decoded state. The existing source-selection pass marks eligible shooters, so unrelated scenes reuse an empty batch without a descriptor call or per-observation output allocation. Recognized shots beyond the horizon remain covered, avoiding fallback prediction. Python reference descriptors remain available through c-shot-descriptors-old.

182 native and 67 planner tests passed. Tests cover every supported phase, both orientations, several accumulator values, horizon filtering and invalid/stale selections. Level-3/4 replay windows matched in both run orders. Level-3 means: 6.97735 ms reference vs 7.01070 ms native; Level-4: 5.56345 vs 5.64329 ms. These small differences do not establish an end-to-end speed benefit (Level-4 reference repeats varied 5.5%). This step migrates descriptor ownership; removing the intermediate generated path copy is measured separately. Reports: world-shots-level3 and world-shots-level4.

Generate scheduled paths directly in shared storage

The existing C shot-path generator now receives its final shared point-buffer region. Body assembly borrows the registered paths instead of copying from a temporary ctypes array. Point forecasts remain cached because all candidate evaluations reuse them; this does not replace useful cached geometry with repeated arithmetic. Reference storage remains available through c-shot-copy-old and when shared buffers are disabled. Borrowed path views retain their owner if the shared pool later grows.

183 native and 67 planner tests passed. The storage test compares every generated point against the old allocation, forbids the registration-time copy, and forces pool growth to check retained-view lifetime. The preceding descriptor migration also passed 575 native-world policy tests. Level-3/4 comparisons matched in both run orders. Direct output removes a temporary point allocation and 2,736 / 1,988 copied bytes per observation on average in those windows. Mean processing: 6.87698 to 6.85638 ms (0.30%) and 5.50237 to 5.45709 ms (0.82%). Both are small relative to repeat variation; no reliable end-to-end speedup is claimed. Reports: shot-storage-level3 and shot-storage-level4 under xenon_tools/run_logs. The other three standard replay windows also matched in single-repeat correctness checks; all five levels are covered. Their timings are not performance evidence.

The remaining Python scene source/group orchestration and body/combat assembly are larger migration boundaries than another small allocation transfer. Missions, planner search orchestration and the complete native controller lifecycle also remain.

C-owned maneuver search (ABI 53, architectural migration)

xenon_autopilot_search.c/.h now owns bounded search ordering, safety-first ranking, retained acceptance, optional attacks, cash excursions, waiting stations, escapes and three-seed repairs. Named request/feedback structs contain no Python types or callbacks. Trajectories remain with the evaluator and are referenced by stable evaluation ids. The fixed proposal bound is derived from the search stages, not from on-screen objects. Three repair seeds are selected without sorting every candidate. Pickup/attack inputs are requested lazily, so accepted retained plans do not perform those selections.

native_search.py is explicitly a transitional evaluator bridge. Native driving uses the C search; ManeuverPlanner.choose_reference() and c-search-old retain the Python algorithm. Scene construction and per-trial evaluation/result adaptation still cross Python. This is not yet a self-sufficient C controller, nor a performance win: single-repeat replay timings were slightly slower with the bridge. The substantial opportunity is to connect search directly to the native workspace and eliminate all per-trial Python masks, prefix keys, motion reservations and result adaptation.

Validation: 185 native tests and 67 planner tests passed; 300 deterministic synthetic scenarios compare evaluation order and winner against the independent Python search, including retained plans, cash, core attacks, clamped duplicate stations, reserve, budget exhaustion and repairs. All five standard replay windows matched in the single-repeat correctness checks (native-search-final-level1 through level5). These timings are not a speedup measurement.

Completion work now targets architectural boundaries rather than small optimizations: connect search/evaluation in C; move scene source and geometry assembly into its C owner; migrate level-specific mission selection and map/mask consumers; supply a persistent native controller lifecycle and transport-free observation/input API. Python reference tests and optional replay inspection remain outside the driving path.

Native search-to-workspace loop (ABI 54)

xenon_autopilot_planner.c/.h connects bounded search directly to the existing candidate workspace. Trial prefixes, forecasts, outcome snapshots, duplicate recipe reuse, lazy combat retries and repair inputs remain in C. Only the selected path is exposed through a borrowed view; losing trials no longer create Python action tuples, prefix-key dictionaries, motion reservations, rankings or Evaluation objects. prepare_request separates observation preparation from per-trial allocation. The existing standalone evaluator and the transitional search bridge remain available. c-planner-bridge-old isolates the bridge; c-search-old compares against the original Python search using the preceding candidate kernels.

The native loop returns only for lazy observation-level cash/attack inputs and first combat capture. Combat geometry preparation and complete scene/mission ownership still need migration. An occupied initial wall position explicitly requests the retained escape adapter before any transition is admitted. Missing native terrain/scene inputs also retain the bridge. These paths are visible in counters; the full controller is still not self-sufficient. Native waiting-side selection uses the contacting path's observed anchor or the center of a linear/sliced hull; it is not guaranteed identical to a separate source object's anchor when those differ.

Validation: 190 native and 67 planner tests passed. Added tests forbid the Python trial evaluator, compare complete winner motions, compare 50 moving/static hazard scenes, check retained one-call execution and verify occupied-start fallback leaves the budget untouched. All five standard replay windows and the active Level-5 barrier window matched in both repeat orders (planner-loop-level1 ... level5, levelbarriers). Means versus c-search-old, in milliseconds per observation:

The substantial gains are Level 4 and active barriers; do not extrapolate to the whole game. The tank window includes inactive/shop observations. Level 2 did not enter the native loop and paid the transitional bridge overhead; resolving its missing native input is the next prerequisite. Level 3 still used occupied-start fallback on about 27.6% of observations. These are replay checks, not live level-completion validation.

Native wall footprints and occupied-start escape (ABI 55)

xenon_autopilot_walls.c/.h completes physical wall motion checks for native planning. It supports exact pixel masks and the existing four-pixel rectangular fallback when a capture lacks the ship collision sprite (the Level-2 recording). Occupied anchors retain the bounded nearest-free/monotonic-distance escape rule. Distance queries are memoized lazily in a small raster/footprint-owned cache; no full navigation grid is built for these probes. Changing observed terrain replaces the owner. Old scenes retain their immutable raster, and map checkpoints isolate mutable native escape caches. A query owner is single-threaded, like the candidate workspace.

The first Level-2 comparison exposed a pre-existing Python sampling bug: intermediate anchors were tested using the hull left at the destination. The missing-sprite fallback therefore inferred changing extents and sometimes an extra obstacle row. At frame 32694 the rightward route was clear at every properly translated sample but the old fallback rejected it. is_player_mask_motion_free now explicitly receives an endpoint hull and translates it along the sampled edge; start-hull callers were corrected. A one-pixel terrain regression checks both the formerly false obstacle and an actual blocking pixel one row below it. The initial 16 Level-2 differences disappear when comparing C to this corrected reference. This is a behavior fix, not forced parity with the defective sampler.

Validation: 194 native, 67 planner, 575 native-world policy tests, and the dedicated terrain sampling regression passed. Native tests compare 500 fractional anchor, distance and edge queries against persistent-map semantics, plus query lifetimes and checkpoint isolation. All five standard windows and active Level-5 barriers match in both repeat orders (wall-queries-verified-level1 ... level5, levelbarriers). Every active observation in these windows now enters the native planning loop.

Versus c-wall-adapter-old with the corrected Python sampler, mean processing times: Level 2 5.8805 -> 3.0931 ms (47.40%), Level 3 6.8539 -> 5.9438 ms (13.28%). Other window differences were small (-1.15% to +2.82%); do not claim gains there. These improvements remove whole Python terrain/eager-forecast fallback paths. There is still Python scene/mission preparation and controller lifecycle work before the complete autopilot can drive from a C observation without Python.

Native optional proposals (ABI 56)

The native planner now handles nearby-pickup and optional attack events internally. XapPlannerTactics borrows the current world plus the scene's existing ordered pickup and supported-target indices. Pickup exclusions come from the candidate scene; combat and tactics reuse the same target selection. Python still supplies mission eligibility until high-level mission policy is migrated.

Attack proposals read the path or linear motion already prepared for collision checks. They do not request another policy forecast, fit velocity again, or sort all targets: C keeps the best two stations with identity as a stable tie-breaker. Targets represented only by conservative swept slices are omitted from optional attack proposals because a swept rectangle is not an exact firing anchor. Required boss/gate missions retain their specialized tactics. Candidate evaluation still checks safety and predicted destruction before accepting a proposed attack.

Validation: 195 native tests and 67 planner tests pass. All six standard replay windows matched the previous proposal bridge (native-tactics-level1 through level5, plus levelbarriers, one comparison repeat). These are correctness checks, not evidence of a meaningful speedup. c-proposal-bridge-old retains the Python selection for ablation. The remaining lazy combat-geometry event and Python scene/mission/controller assembly still prevent a standalone C autopilot.

Shared native prediction owner (ABI 57)

XapPredictions now owns the observation's shared point pool, path slots, and recursive dispatch for captured script roots, ordinary delayed followers, same-frame follow/attachment links, coupled composites and shell oscillators. The Python shared-scene adapter borrows these buffers; body/combat preparation still addresses the same point indices. A tail request resolves and prepares its root in C, and later member requests reuse that root. Script roots are prepared on demand instead of forecasting every captured script at the first request.

Completed point views remain valid across pool growth. C retains old allocations until the observation owner is released; unused reservation tails can be rewound but completed native paths cannot be overwritten. Native dispatch rejects changed observation identities, changed camera conventions and cyclic links. The world must outlive the prediction owner and remain unchanged while queried.

The existing Python-owned pool and dispatch remain available with c-prediction-owner-old. Generic/specialized roots still return unsupported and use the existing bridge. Articulated eye/boss assembly and the independent shell consumer have not yet been unified with this dispatcher. Consequently this is shared ownership and recursive dispatch migration, not full C scene preparation.

Validation: 197 native tests, 67 planner tests, and all six replay windows match (native-predictions-verified-level1 through level5, plus levelbarriers, one repeat). Unit coverage includes complete recursive path comparison against the existing C kernels, repeated root reuse, cycle rejection, protected reservation rewinds and old borrowed views after growth/longer forecasts. The short timings do not establish a substantial speedup; this stage removes a C runtime dependency.

Independent roots and shared eye/boss paths (ABI 58)

The shared C dispatcher now owns ordinary fitted-velocity roots, articulated eye assembly and delayed eye/Level-4 head batches. Linked/composite requests can resolve an independently moving root entirely in C. Fits occur only when such a root is actually requested. The independent directional-projectile root remains unsupported here until its exact direction/scale capture is retained in C.

Eye and boss consumers now borrow the same pool and path slots used by recursive links. The existing kernels are reused; this does not introduce a second script interpreter or a second eye model. Python still exports requested eye dictionaries for policy consumers. They will become optional observation views after mission policy migration. General scene classification, specialized collision envelopes, combat-only blockers, and high-level controller/mission assembly remain Python migration dependencies.

Validation: 198 native tests pass, including independent-root history comparison and existing articulated/boss differential suites. All six replay windows matched (native-root-dispatch-level1 through level5, plus levelbarriers, one repeat). Timings were similar; no substantial speedup is claimed for this ownership step.

Captured directional roots (ABI 59)

The native object decoder retains validated signed direction-table index and speed scale. Shared root dispatch uses the exact 16-direction game table when these fields are present; malformed or incomplete captures retain the ordinary history fallback. A known directional projectile never needs velocity fitting. Python world updates now consume the common decoder result instead of parsing these words again. The Python decoder remains the reference implementation.

Validation: 199 native, 67 planner, 575 forced-native policy and 6 decoder tests pass. Direction tests cover all 16 headings at positive/negative speed with misleading history and prohibit the redundant Python decoder call. All six replay windows match (native-direction-roots-level1 through level5, plus levelbarriers, one repeat). No substantial speedup is claimed. Remaining specialized collision envelopes and scene/mission assembly still require migration.

Native animated collision envelopes (ABI 60)

XapEnvelopes prepares complete collision-sweep batches from the captured world. Level-4 laser jaws and diagonal sweepers live in xenon_autopilot_level4_envelopes.c; tank growth/fall lives in xenon_autopilot_level5_envelopes.c. The world now retains the observed update handler alongside status so these dispatchers need no Python classification or raw-byte adapter. Generated bounds stay C-owned and enter the existing native slice packer through borrowed pointers.

Each model advances once through the horizon. Diagonal sweepers request the shared generic root forecast only if the horizon reaches the reset boundary; laser and tank models need no fitted forecast. Post-reset sweeper behavior and post-trigger jaw RNG approximations retain the established reference rules. Candidate camera correction remains with the existing source metadata.

Validation: 202 native tests and 6 decoder tests pass. Targeted tests compare 80-frame sweeps across orientations, phase/accumulator states, reset boundaries, tank extent and vertical speed, and prohibit Python hull prediction in the production slice adapter. Six replay windows match (native-animated-envelopes- level1 through level5, plus levelbarriers, one repeat). The tank window uses the new path; standard Level-4 wall windows contain no admitted animated source, so the targeted differential tests provide jaw/sweeper coverage. No substantial whole-window timing gain is claimed. c-animated-envelopes-old retains the old shape preparation. Level-3 animated/reflecting/expanding shapes, safety envelopes, combat-only blockers and full mission/controller assembly remain to migrate.

Level-3 collision envelopes (ABI 61)

C now prepares animated attachment/projectile collision headers, exact reflecting projectile fractions and the conservative expanding-formation hull in a dedicated Level-3 module. Sprite headers are generated by generate_native_envelope_data.py from the extracted Python asset tables. Formation words are decoded once by the Level-3 decoder, rather than reparsed during each world update. Known overlays now restrict formation decoding to Level 3; unknown-level decoding remains supported. Python retains the reference implementations and optional observation views.

Validation: 206 native, 67 planner, 575 forced-native policy and 6 decoder tests pass. Differential tests cover all animation frames/countdowns, reflection turn boundaries and signed fractions, and controller/follower formation states. All six standard replay windows match (native-level3-envelopes-*, one repeat). The short diagonal-sweeper recording also matches. These runs do not establish a substantial speedup. A separate older composite recording cannot complete even with the old envelope backend: the Python mission controller selects the minimum of an empty tactic-constrained action set. This is a remaining controller defect, not evidence of envelope parity for that recording.

Remaining driving dependencies include special safety envelopes, pending synthetic sources, lazy combat scene assembly, mission/controller policy and transport. The autopilot is not yet self-sufficient in C.

Lazy world-owned combat assembly (ABI 62)

XapWorldCombatInput supplies canonical source indices and flags, the shared prediction owner, target selection and damage bonus. The planner now requests health, player bullets and combat geometry inside C only when a trial needs combat. The workspace owns frame ranges and endpoint buffers; unchanged body paths and linear descriptors remain borrowed from the same scene. Tank growth and expanding fronts reuse their level-specific envelope functions. Ordinary sources use shared root paths, and offscreen/unscored blockers retain the conservative union with their observed hull.

Python no longer constructs CombatScene or primes per-source prediction for this path. Pending synthetic sources and legacy ablation configurations retain the compatibility callback. The optional workspace status API lets Python tests and inspection reuse C-prepared health/results without a second initialization. No Hatari integration was added; the API remains callable from Python.

Validation: 211 native, 67 planner and 575 forced-native policy tests pass. Targeted tests compare 80-frame endpoints and blocker unions, exercise Python inspection after C preparation, and prohibit the Python combat callback during a native planner session. All six replay windows match in both execution orders (native-world-combat-isolated-*). Levels 1/2 reduce mean planner calls from 1.16/1.148 to 1 per active observation. Timings are largely unchanged; the tank window improves about 6%, not a substantial overall speedup.

Earlier native-world-combat-level5 and native-world-combat-verified-* timings are INVALID: the first comparison variant unintentionally disabled other native optimizations. Only the corrected isolated reports support this comparison.

C still needs complete scene/source assembly, pending/safety models, mission/controller policy and the driver boundary before standalone operation.

Composite replay arbitration repair

The older composite recording exposed an empty preferred action set when its latched staging gap became unreachable. Survival arbitration now falls back to physically legal actions (or the complete action set if all are blocked) instead of calling min() on an empty set. This does not relax collision scoring.

Validation: the new targeted regression and all 575 policy tests pass. The 203-frame composite recording now completes and both combat variants match (native-composite-empty-tactic-fixed, frames 40321-40523).

Timed emitter safety zones (ABI 63)

The Level-2 decoder now captures validated emitter age once. C stores missing age as -1 in a typed field (valid ages are 0..20); Python exports it only when present. The world update no longer invokes a second raw-byte age parser. Known overlays restrict this handler to Level 2; captures without level metadata remain supported.

The level-specific envelope module reserves the launch area from the predicted spawn frame through its reaction window. It does not fit or invent an enemy trajectory. The existing C envelope owner and slice packer consume these bounds; include_safety makes the policy zone explicit and the old adapter remains an ablation. Replay counters distinguish safety from animated-envelope sources. The obsolete WORM_HOLE enum branch is not emitted by current classification; current wormhole objects use Level-2 boss-eye classification. Its legacy Python branch was not duplicated in C.

Validation: 212 native, 67 planner, 575 forced-native policy and 6 decoder tests pass. Targeted tests cover every launch-window boundary, invalid ages and short captures while prohibiting duplicate Python age parsing and safety prediction. The Level-2 homing recording matches (native-emitter-envelopes-level2, one repeat). This is a dependency-removal step; no substantial speedup is claimed. Full scene assembly and mission/controller/driver migration are still required.

Canonical body-scene assembly (ABI 64)

xap_world_bodies_create assembles canonical source selections into a complete C-owned XapManeuverScene: analytic linear bodies, shared root paths, animated or timed safety slices and frame ranges. Constant-velocity sources stay analytic; they do not allocate horizon-sized point arrays. The scene can be passed directly to C candidate preparation. Python's current adapter borrows descriptors and translates source identities; frame labels are lazy instead of replicated lists.

The factory deliberately retains the compatibility assembler for scheduled-shot sources, synthetic spawns, collapsed chains and destruction transitions. The last case exposed a still-Python world rule: an enemy installing its destruction handler remains hazardous for one observation. Treating that as ordinary motion extended its body through the horizon. The explicit guard restores the established rule; its native world-state migration is a follow-up dependency.

Validation: 215 native and 67 planner tests pass. Targeted tests forbid Python hull and anchor preparation, check analytic collision queries against Python, verify C safety frame ranges/lazy labels, and protect destruction-transition routing. All six replay windows match: native-world-bodies-destruction-guard-level1 and native-world-bodies-guarded-* for the remaining levels/barriers (one repeat). The earlier native-world-bodies-level1 report contains the now-fixed destruction regression and must not be treated as the final result. Timings are similar; this is an ownership/dependency migration rather than a substantial measured speedup.

Native driving still needs the remaining source assemblers, complete world postclassification, mission/controller policy and the driver boundary.

Native destruction transitions and lazy track checkpoints (ABI 65)

C tracks retain current and previous semantic kinds. Classification preserves hostile contact for exactly the first destruction observation, including after checkpoint restore or a Python-to-C backend switch. Repeated decoding uses the stable previous kind. The normal native world path skips Python's transition postprocessing; the reference/ablation path remains available.

The canonical C body scene now owns the final stationary contact hull and leaves empty collision slots after that frame. It never extrapolates old scripted velocity through the destroy handler. This removes the destruction-transition scene fallback. An intermediate replay found that extrapolation mistake; the stationary implementation is covered by a moving-history regression.

NativeWorld no longer copies every track and its twelve history samples into a checkpoint buffer on each update. Ordinary compatibility headers are read directly from C storage. Immutable snapshots are copied lazily when requested, after classification, and remain safe across replay forks and restores.

Validation: 216 native, 67 planner, 575 forced-native policy and 7 decoder tests pass. Tests prohibit eager snapshots, restore semantic history, check every hostile-transition kind, and verify stationary one-frame collision geometry. The Level-1 comparison matches (native-world-destruction-stationary-level1, frames 2367-3367). The earlier native-world-destruction-level1 report records the fixed velocity-extrapolation error. No substantial timing gain is claimed. Scheduled-shot/collapsed/synthetic scene assembly and mission/controller/driver migration still prevent fully independent C driving.

ABI 66: scheduled enemy shots assembled in the shared C scene

The native body factory now appends Level 3 small-wall and Level 4 directional-wall shots directly to the shared prediction pool. Python no longer builds their shot entries on this route. Spawn frames and independent lifetime after shooter death are retained; initialized shot spans are protected from reservation rewind. Other wall-shot families and noncanonical sources still use the reference bridge. Validation: 217 native and 67 planner tests pass. Recorded Level 3 small-shot and Level 4 wall windows match all verdicts (native-scene-wallshots-level3/4). Single-run means were 5.972 to 5.786 ms and 4.115 to 4.020 ms respectively; these are migration checks, not evidence of a substantial performance improvement.

Composite body scene assembly (ABI 66, unchanged layout)

The C scene factory now accepts composite chain members. Dormant chains use the existing native collapsed-bound kernel directly from native root/ordinal metadata; only the emitter path is expanded. Active chains use the shared coupled predictor. Python no longer supplies collapsed rectangles to this scene. This reuses existing kernels rather than introducing a second prediction model. Validation: 218 native and 67 planner tests pass; the 203-frame composite replay matches all verdicts (native-scene-composites-level3). No substantial performance gain is claimed.

Level 3 wall-shot envelopes (ABI 66, unchanged layout)

The C body factory owns the remaining Level 3 oscillator and large wall-shooter shot envelopes. The large-gun forecast advances its finite firing states once through the horizon, retains both RNG phase outcomes, and stops each branch at its first allocation. Each resulting bullet advances once through its remaining frames. This replaces Python's repeated reconstruction for every requested frame. Shot slices retain independent lifetime and exact initial-update/reflecting rules. The Python implementations remain as comparison oracles.

Validation: 219 native and 67 planner tests pass, including all 18 large-gun phases at three accumulators and oscillator pause/reset boundaries over 80 frames. The 270-frame large-gun recording matches in both run orders: native-level3-shot-envelopes-large-repeat. Mean time fell from 6.882 ms to 4.814 ms (30.0%). This is a workload-specific gain, not a whole-game estimate. The older oscillator recording cannot yet complete the baseline: native workspace selection rejects a noncanonical target. Its failed report is not validation of the shot migration; that compatibility boundary needs correction separately.

Noncanonical combat-target boundary

Native workspace creation now resolves the target selection before allocation and reuses it for tactics/combat. Predicted targets lacking a captured world slot keep the compatibility owner instead of failing later with a stale-selection exception. Validation: 220 native and 575 forced-native policy tests pass. The previously blocked oscillator recording now completes and matches all verdicts (native-level3-shot-envelopes-oscillator-canonical, 554 frames).

ABI 67: canonical candidate-shot assembly

xap_prepare_world_candidate_scene accepts canonical periodic/composite source indices, selects captured spawn delays in C, requests shared root paths, and builds candidate fire descriptors without Python anchor calls or output copying. It uses the existing candidate-scene factory and evaluation kernel. The Python adapter keeps only source identities and the explicit unsupported-root fallback. The API is callable by either a C driver or ctypes and is exported for WASM.

Validation: 221 native and 67 planner tests pass. A direct-scene regression forbids Python shot-source construction and compares periodic, radial and composite shot descriptors. Composite, stage-2 and Level-5 replay verdicts match (native-world-shot-assembly-*, one repeat); timings are effectively unchanged. This removes another assembly dependency, not the remaining high-level mission, synthetic-source, world-input and driver dependencies.

ABI 68: complete native waypoint connectors

xap_span_connector owns world-to-grid rounding, nearby legal-anchor selection, start-escape limits, span search, corner-safe line tests, bounded smoothing and selected world-route output. It shares the existing graph and fixed scratch, with no Python nearest-anchor or smoothing callbacks. The span backend uses it by default; c-connectors-old retains the complete previous connector for comparison. Grid preparation and high-level mission selection remain separate dependencies.

Validation: 222 native, 67 planner and 575 forced-native policy tests pass. Randomized complete-route tests cover blocked endpoints, camera row limits and rounding while prohibiting Python nearest/smoothing calls. Recorded Levels 1-5 match all verdicts (native-world-connectors-level*, one repeat). Timing is essentially unchanged; no substantial performance gain is claimed for this step.

ABI 69: homing-launch origins inside the native workspace

xap_workspace_create_predicted resolves future launcher anchors from the shared C prediction owner before initializing homing reset state. Canonical driving no longer calls Python's anchor predictor for those allocations. The previous host-supplied-anchor API remains available for reference and unsupported roots. Both entry points share one initialization implementation.

Validation: 223 native and 67 planner tests pass. A regression prohibits Python anchor prediction and compares complete initialized missile state. The Level-5 recorded verdicts match (native-launcher-roots-level5, one repeat); timing is unchanged. No WASM profiling or Hatari integration was performed.

ABI 70: complete canonical world-scene owner

xap_world_scene_create assembles independent bodies, shell releases, periodic and composite fire, barrier volleys, homing reset state and lazy combat inputs in one C operation. It owns the candidate scene and workspace; callers may reuse a compatible body owner or let C build it. World, predictions and wall-query data remain borrowed. Python retains selected identities and optional result labels. The previous component assembler remains available as c-world-scene-old.

Validation: 224 native, 67 planner and 575 forced-native policy tests pass. Tests prohibit Python component assembly and cover both borrowed and C-created body ownership. All verdicts match across recorded Levels 1-5 and the additional barrier window (native-whole-world-scene-*, one repeat). Timings are essentially unchanged; this is an ownership migration, not a claimed performance gain. High-level mission policy, source selection, synthetic inputs and the game driver still prevent standalone operation. No Hatari integration was performed.

Native body handoff without eager Python inspection

The canonical body adapter now passes the existing C header directly into the maneuver environment and complete scene factory. It no longer invokes the empty Python body assembler, validates an empty placeholder scene, creates geometry array views, resolves every contact label, or allocates compatibility query buffers during handoff. Those views, labels and query buffers are lazy; explicit inspection and the retained Python-driven query path remain available.

Validation: 225 native, 67 planner and 575 forced-native policy tests pass. A regression forbids both the old body constructor and geometry-view creation while building the complete native scene, then verifies optional inspection. Level-1 and active Level-5 recorded verdicts match component assembly (native-lazy-body-handoff-*, one repeat). No substantial timing gain is claimed.

ABI 71: shared native scene selection

xap_scene_selection_create owns source classification, identity-sorted damage targets, per-source target slots, independent-body selection, unmodeled player-fire blockers and pickup indices. Body and complete-scene preparation borrow these arrays. The complete scene also reuses the captured source flags instead of classifying the same sources again. The API takes canonical scored indices; the high-level policy deciding which hazards to score is still a separate dependency.

Removed work includes Python target sorting, world-wide blocker membership scans, repeated canonical index conversion and source-structure reconstruction between selection, body preparation and complete-scene preparation. Python's damage-support membership shares the target dictionary, and Python blocker objects are resolved only if a compatibility consumer requests them. Identity inspection retains the original observation; stale selections cannot be handed to a newer native world. The old path remains available as c-selection-owner-old.

Validation: 228 native, 67 planner and 575 forced-native policy tests pass. Tests cover reordered source subsets, actual target sorting and nonempty blocker/pickup sets, duplicate rejection, retained identity lifetimes, and handoff with repeated Python index conversion prohibited. Recorded Levels 1-5 and barriers match all verdicts (native-selection-owner-*). Opposite-order repeats after lazy blocker resolution also match (native-selection-owner-lazy-level1/level3): mean time is 2.845 -> 2.886 ms on Level 1 and 4.634 -> 4.569 ms on Level 3. These mixed changes of roughly 1.5% do not establish a meaningful speedup. This removes another host assembly dependency; it does not make the complete autopilot C-only.

ABI 72: native player/camera planning origin

xap_prepare_player_state reads the canonical player and scalar telemetry to initialize ship coordinates, steering, speed, camera intent/limits, pending input, the observed/next-update collision union and the three steering-dependent hulls. Protocol-presence flags retain historical camera and collision-bound fallbacks. The result is a plain caller-owned struct; it requires no allocation, retained world pointer or Python geometry callbacks and can be consumed directly by C.

The Python driver packs captured values once per prediction service. Native maneuver preparation reuses the result instead of invoking camera/hull helpers, rebuilding steering hulls and recomputing the initial rectangle union. The old initializer remains available through c-player-state-old; optional Python camera and rectangle views support the existing evaluator/replay interface.

Validation: 230 native, 67 planner and 575 forced-native policy tests pass. Tests compare 96 combinations of telemetry availability, steering, fractional legacy scroll velocity, limits, sprite/global collision data and pending input, and prohibit Python camera/hull callbacks during native consumption. Invalid telemetry leaves output untouched. All recorded Levels 1-5 and barrier verdicts match (native-player-state-*, one repeat); timings are essentially unchanged. This closes player-state initialization, not mission policy, synthetic-source preparation, equipment policy or the complete game driver. No Hatari integration or WASM profiling was performed.

ABI 73: native player-fire schedules and phase reuse

xap_prepare_fire_schedule takes a pulse configuration and normalized phase and writes intended shot times plus the current Atari joystick fire bit into caller-owned storage. It is a plain C driver API; cooldown and weapon allocation remain separate game behavior, so these times are fire intent rather than proof that a bullet exists.

The live driver and replay use the same bounded phase cache. The normal period-2 pulse needs two schedules, which survive firing-step resets. Native world-scene, component-scene and fire-descriptor preparation borrow those buffers instead of packing them again. Replay also retains its FirePulse configuration. Explicit custom shot-time overrides retain the old packing path, and c-fire-schedule-old preserves per-frame reference schedule generation.

Validation: 234 native, 67 planner, 575 forced-native policy and 24 replay tests pass. Pulse tests cover disabled/held fire, period and phase boundaries, negative driver steps, horizons, delays, invalid capacities and custom overrides. The default pulse requires only two C builds across 104 steps including resets, with Python pulse calls prohibited. All recorded Levels 1-5 and barrier verdicts match (native-fire-schedule-*, one repeat). Single-pass timings show no meaningful overall improvement; this removes repeated driver work and another C-driver dependency. No live gameplay, Hatari integration or WASM profiling was performed.

ABI 74: complete scene-to-decision session

Migration now follows subsystem boundaries instead of adding a Python wrapper for each small C helper. xap_decision_run owns scene creation, derivation of the search/maneuver/evaluation requests, transition-budget storage, optional combat and detour evaluation, winner selection and retained-plan updates. These steps execute in one C call without host callbacks. XapDecisionSession retains only a bounded action sequence, age, mission identity and core-attack return station; it never retains a previous world or assumes a suffix remains safe.

The host passes one semantic mission, configuration, prepared player state and external scene inputs. A mission identity counter invalidates retention when a level/tactic/target/goal changes. Discontinuous frames, reset/restore/shop, refresh age and unsafe winners also prevent reuse. C revalidates every retained suffix against the current world. The session is independently owned from each returned XapDecision, so a winning path remains readable after later decisions or session release. World, shared predictions, wall queries and reused body storage retain their existing observation-scoped borrowed lifetimes.

Waste removed from normal native decisions: - Python assembly of three overlapping planner request structs and tactic/combat handles. - Python allocation/packing of retained action suffixes and the transition tree. - Previous-winner motion certificate reconstruction, rectangle/camera comparison, motion-cache population and prefix seeding before C recomputes the trajectory. - Host scene/workspace inspection merely to connect one C subsystem to another.

Python's native_decision.py is the single subsystem adapter. The preceding component assembler remains separately selectable as c-decision-session-old; unsupported scene/wall inputs still use it and increment native_decision_fallbacks. Clear winners do not request a scene view. Tests and replay can explicitly obtain service.native_candidate_scene, which lazily borrows the decision-owned scene. Existing winner diagnostics still materialize Python motions in the controller; this change does not claim all observability or all observation preparation has left the driving path.

Validation: 239 native, 67 planner, 575 forced-native policy and 24 replay tests pass. Targeted tests compare complete winning paths over initial positions and pending inputs, prohibit old request builders/scene inspection during a clear native decision, cover retained refresh/reset/discontinuity/mission changes, check result lifetime and rejected-input behavior, and exercise compatibility.

Recorded comparisons (two opposite-order repeats, pinned CPU) all match safety verdicts. Every measured native observation used the new decision entry point. Reports: xenon_tools/run_logs/native-decision-session-{level1,level2,level3,level4,level5,barriers}/comparison.json.

Only Level 1 demonstrates a substantial improvement here: its planning portion falls from about 0.754 to 0.532 ms. Transition counts increase because the native budget now counts prefixes formerly pre-seeded by Python certificates; rollout counts and safety verdicts match. The bounded search continues to honor its configured budget. These are recorded counterfactual comparisons, not new live campaign validation or mobile/WASM measurements.

Remaining large boundaries: canonical observation/equipment/pending-spawn input assembly; high-level mission selection and level-specific tactics; and the game lifecycle/transport/shop driver. Prefer migrating those whole consumers next, reusing this decision API directly. Standalone C callers no longer need Python to connect scene preparation to planning, but still must supply the mission and external observation inputs. No Hatari build integration was added.

ABI 75: complete captured-observation normalization

The next whole subsystem is the captured-frame input to the canonical C world. xap_world_observe accepts raw object headers/anchors, captured byte spans, draw events, destroyed identities and frame/camera metadata. C filters the live population, matches draw identity/address pairs, merges render rectangles, extracts visible walls, selects coordinate rules, decodes collision words, and runs tracking, classification and enabled prediction-model capture together. The old host-prepared tracking/decoder APIs remain usable for tests and the c-observation-old comparison; the production path has one observation call.

Waste removed from the native path: - Separate Python coordinate-policy and collision-bound preparation per object. - Python live-draw key construction and rectangle-union intermediates. - Separate tracking/field-decoder metadata batches and repeated record traversal to prepare the second C call; one captured object batch/raw payload serves both. - Fresh observation preparation buffers inside C after capacity has stabilized. - Temporary ctypes geometry wrappers during required compatibility export; borrowed rows are read in bulk instead.

The C world owns observation scratch and returns borrowed live-record mappings, tracks, fields and wall views. It retains needed prediction data independently of the temporary captured input arrays. Python still exports TrackedObject, script and wall views because current high-level tactics/replay consume them. Equipment telemetry interpretation, synthetic pending-source policy and mission selection are not part of this port. Their consumers must move before normal Python-hosted driving can omit compatibility export entirely. Native callers can already invoke observation normalization without any Python logic or callbacks.

Validation: 244 native, 67 planner, 575 forced-native policy and 24 replay tests pass (910 total). Targeted coverage compares changing populations and geometry with the component pipeline, exercises every named handler's coordinate policy across levels, prohibits the old Python preparation/second decoder call, and checks the direct C API, empty frames, capture-buffer reuse, 64-bit identities, stale/duplicate draws, visible walls, namespace resets and invalid raw spans.

All six recorded safety comparisons match in both run orders. Final reports are xenon_tools/run_logs/native-observation-bulk-{level1,level2,level3,level4,level5,barriers}/comparison.json. These follow the removal of temporary ctypes export wrappers; earlier native-observation-* reports measure the initial adapter.

There is no meaningful overall speed improvement to claim. The architectural result is removal of another complete Python preparation dependency, with small mixed timing changes from -1.6% to +0.8%. The current JSON driver must still pack captured draws and materialize legacy consumers' views. Further substantial work should migrate those consumers together with high-level mission policy, not add more isolated scalar wrappers. No live campaign, Hatari build integration or WASM profiling was performed.

ABI 76: Level 1 mission selection and common navigation

High-level mission selection now has a separate C entry point, with common route ownership in xenon_autopilot_mission.c and an explicit encounter priority in xenon_autopilot_level1.c. It consumes the current C world and shared predictions directly and returns the existing evaluator's mission structure. The supported native/span Level 1 path no longer calls ReactiveController.choose, updates unrelated Python level-state machines, or maintains duplicate Python shell stations. Levels 2–5 retain the reference selector for the next migration stages.

Final opposite-order recorded comparisons reduce Level 1 decision time by 32.2% (2.474→1.677 ms) in the early sample and 26.7% (3.084→2.260 ms) in the shell/boss sample. Other levels retain identical decisions; their timings are effectively unchanged. 919 tests pass. Live checks reached both tested shops with no lives lost: the first segment took one 4-point hit, while the shell/boss segment took zero damage and completed more slowly than the archived reference run.

See mission architecture and validation for source boundaries, retained Python dependencies, recording paths and limitations.

ABI 77: complete Level 2 mission module

Level 2 now selects its authored backbone, ordered eyes, serialized homing waves and final-boss entry/side lane in C. Common mission routing is shared with Level 1. See AUTOPILOT_MISSION_MIGRATION.MD for validation and remaining host dependencies.

ABI 78: Level 3 gate and carrier missions

The Level 3 mission consumes native persistent group state and shared predictions directly. Restored group IDs retain embedded ordering. See the mission migration document for the separate recorded and bounded live validation results.

ABI 79: Level 4 missions

Wall-shooter stations and both boss phase controllers now run in C. Route intent and corner-safe waypoint advancement are shared. The bounded live comparison reproduced a navigation pocket stall in both native and reference missions; see the migration document. The final recorded comparison is 3.5% slower, not a performance win.

ABI 80: all level missions native

Level 5 progression and tank tactics complete the per-level mission migration. Shared native region searches replace individual firing-station trials and reuse graph scratch. No per-tactic Python callbacks were introduced. Observation/host assembly and lifecycle remain separate migration work; C is not yet a standalone game driver. See AUTOPILOT_MISSION_MIGRATION.MD for measurements and gameplay limitations; enabled native missions do not imply whole-campaign acceptance.

ABI 81: C-owned driving coordinator

xap_driver_step now owns source selection, shared prediction, pending Level 3 script paths, mission state and evaluation/retention. The active native/span path bypasses Python prediction/planner objects and per-frame mission-state copying. Map environment preparation is retained across unchanged observations. Detailed replay inspection is explicit; losing trial storage does not survive in snapshots.

Additional recorded processing reductions range from 26.9% to 59.5% across six samples compared with the preceding C-mission/Python-coordinator implementation. Two bounded live checks took no damage and lost no lives. See native driver architecture and validation for exact measurements, recordings, API ownership, and remaining host/lifecycle dependencies.

ABI 82: native navigation environment

The shipped-assets path now prepares raster, cropped/dilated ship footprints, span graphs and wall queries directly from native map cells. Recurring footprints share a raster and a bounded native geometry cache. Span preparation skips unused clearance calculation and Python grid export/reimport. Collision masks join the manual asset generator. Fresh preparation improves 52–73%; cached recorded frame time is approximately unchanged. See AUTOPILOT_NATIVE_ENVIRONMENT.MD for ownership, measurements, validation and remaining observation/lifecycle dependencies.

ABI 83: native map observation session

Persistent map seeding, frame/identity reset rules, visible-band updates, snapshot RAM synchronization and checkpoint cloning now run in C. Native wall draws stay inside C; Python wall objects, map cells and resident metadata are lazy views. Recorded processing improves 5–8% with matching decisions in three samples. See AUTOPILOT_NATIVE_MAP_SESSION.MD for validation, API ownership and the remaining object-world compatibility bridge.