Autopilot · write-up
Native canonical object tracking
Scope and ownership
ABI 19 introduces a transport-independent XapWorld context. It owns the live
canonical population, identity lookup, normalized coordinates, selected collision
bounds, velocities/accelerations, age, and the last 12 canonical motion samples.
Absent identities disappear from the next population. Namespace/run changes and
non-increasing frames reset continuity. Reused object slots retain no history
unless the canonical identity also matches.
Post-capture game-memory telemetry can replace the player's collision rectangle. The next batch supplies that corrected prior shape explicitly, preserving the same stale-word fallback as Python. Switching a running world from Python to C imports the previous population once; ordinary native frames never re-import it.
The context retains growing population buffers and a hash lookup across updates.
One batch advances all live records; there are no per-object C calls or Python
callbacks. XapTrackInput, XapTrack and XapTrackSample have named C fields and
matching ctypes structures. C performs player stale-collision fallback, bullet
render-bound selection, pickup stale-bound selection, tile-anchor normalization,
and update-pass/draw-pass scroll handling.
Python still performs authoritative lifecycle filtering, render aggregation, handler-to-coordinate-policy mapping, game-specific field decoding, classification, and tactics. This is a first ownership slice, not a completed WorldState port. In particular, the input to the new API is the canonical live population, not unfiltered object observations or TCP messages.
Compatibility and observability
Python compatibility headers are decoded in bulk rather than through repeated
ctypes rectangle proxies. Python TrackedObject values remain available to existing tactics and replay.
C retains independent motion history. The current compatibility view extends
Python history using the previous immutable tuple, avoiding a costly full export
of twelve samples per object. This duplication remains until Python consumers
can read native state directly or request diagnostics lazily.
Replay copies share immutable native checkpoint bytes. Only a resumed checkpoint
allocates/restores a context. Context destruction uses a Python finalizer calling
xap_world_free; C allocates no Python objects. TCP and Hatari integration remain
unchanged. WASM exports are maintained without WASM profiling.
Raw object hex was previously decoded by several independent field readers. The reuse stage decodes once per record and passes immutable bytes to those readers. The old decoding path remains selectable for measurement.
Measurement controls
c-world-old: Python tracking and original repeated raw decoding.c-world-bytes: Python tracking with raw-byte reuse.c: raw-byte reuse and the native tracking owner.
Final repeated timing
Level 1 frames 2367–10059 (7,693 observations), sequential workers pinned to CPU 4, two repeats in opposite variant order:
| Variant | Mean observation time |
|---|---|
| Python tracking, original raw decoding | 3.19246 ms |
| Python tracking, reused raw bytes | 3.16472 ms |
| Native tracking, reused bytes, bulk compatibility headers | 3.20598 ms |
Raw-byte reuse saves 0.87% in this run. Adding native tracking to that stage costs 1.30%; the net result versus the original is 0.42% slower. That net difference is smaller than the baseline's 1.22% repeat spread: no demonstrated overall speedup. C remains selected by the explicit C kernel backend for the ownership migration; Python remains independently selectable. This is not a completed WorldState port.
The initial bridge used individual ctypes field/rectangle proxies. Its two-repeat mean was 3.2411 ms, versus 3.1869 ms for its own baseline. The final bridge unpacks fixed headers in bulk, skipping the twelve retained history samples. Treat the separate runs as diagnostic evidence about overhead, not a precise isolated percentage improvement for the bridge.
Final timing report: xenon_tools/run_logs/world-tracking-final-timing-0908/comparison.json.
Initial bridge report: xenon_tools/run_logs/world-tracking-timing-0908/comparison.json.
Both timing comparisons have zero changed decision verdicts.
Profile and validation
A separate instrumented old/new replay measured WorldState.update cumulative
seconds of 8.4825 versus 8.3274; the new wrapper accounts for 0.8488 seconds of
the latter. These profile times locate costs and are not speed estimates.
Reports: world-tracking-profile-0908/comparison.json and its .pstats files.
The full suite passes 928 tests, including four new tracker tests. Additionally,
all 575 existing WorldStateTests pass when their update calls are forced through
C. Targeted coverage includes every tracked field, C-owned history, discontinuous
frames/namespaces, slot reuse, destroyed identities, restored checkpoints, backend
switching, corrected player geometry and malformed raw bytes/batch arguments.
Old/new decision verdicts match in both Level 1 timing repeats, the profile run,
and checked Level 2 (961 observations), Level 3 (225), Level 4 (900), and Level 5
(5,652) recordings. These are recorded-observation comparisons, not fresh live
closed-loop runs. Cross-level reports are under
xenon_tools/run_logs/world-tracking-level{2,3,4,5}-0908/comparison.json;
full test log: xenon_tools/run_logs/world-tracking-tests-0908.txt.
The standalone library was built with hatari_dev.ps1 -Action build-kernel.
No emulator integration or WASM profiling was performed.
Native versus compatibility cost
A separate diagnostic pass wrapped the native batch call and the complete Python bridge with timers over the same 7,693 observations:
| Component | Total seconds |
|---|---|
| Native update call, including ctypes call overhead | 0.04251 |
| Python bridge excluding that call | 0.45582 |
| Combined bridge | 0.49833 |
About 91.5% of this bridge's measured time is outside the native call. This does
not include the subsequent TrackedObject constructors or game-specific Python
field decoding. The timers themselves add overhead; use the repeated timing
experiment for overall speed, and this diagnostic only for cost attribution.
Artifact: xenon_tools/run_logs/world-tracking-bridge-0908.cost.json.
The next useful migration is native consumption and game-specific field preparation over the retained tracks. Eliminating the mandatory Python object view and repacking is more valuable than micro-optimizing the now-small C update.