Xenon 2

Behind the scenes

The autopilot

A controller that plays Xenon 2 by itself — written almost entirely by an AI coding assistant, under the direction of a human who kept score of what went wrong.

What it is

Every emulated frame, the emulator hands over a canonical snapshot of the game's memory: every live object with its position, type, sprite and update routine, the tile map, the camera, the ship's exact sub-pixel state. From that the autopilot maintains a world model — a persistent map of walls it has seen, tracked enemies with predicted motion, destructible gates, pickups worth collecting — plans a route through the level, picks a tactic (advance, retreat, intercept, suppress, escape) and presses the joystick.

It started as a Python program talking to the desktop emulator over a socket, one frame at a time. As the observations and tactics settled, the hot paths were moved into dependency-free C that now runs inside the emulator itself, with the Python kept as a reference and an inspector.

Where it stands

It finishes the game. On 5 October 2026 four uninterrupted runs flew from Level 1 to the destroyed Level 5 ship without losing a life, with level-specific route, boss and shop policies (the runs). It still takes damage, which pickups and shop repairs make good, and it still misses some pickups and a boss reward.

Getting there took far longer than it should have. It took countless experiments before each level could be completed, and the AI kept repeating the same mistakes. Most of the trouble was terrain and navigation: for weeks the ship got stuck in front of open passages for no visible reason, or kept bumping into the walls, while the AI patched one waypoint or one wall rule after another. Where the ship stopped, a valid route was often already there; it was lost on the way from the plan to the joystick, and no amount of waypoint patching could fix that (the navigation review).

Why the mistakes are the interesting part

The retrospective is the most honest document on this site. It records, in order, every wrong turn the AI took while building the controller: guessing values that could have been measured, mixing screen and world coordinates, letting a pickup silently take over the joystick, patching symptoms of a broken planner contract, declaring success before replaying the failing frame. Again and again the entries come back to walls, terrain and routes. Each entry names the root cause, and the patterns section distils them into rules that changed how the rest of the work was done.

228 ways an AI got it wrong

The full retrospective, with a filter box. Some highlights:

  • #11 — Used screen-space motion in a world-space game model
  • #20 — Pickup collection had unconditional priority until danger was already immediate
  • #44 — Declared dead-end fixes successful before replaying the reported failure
  • #90 — Invented wall occlusion that the game's projectile code does not implement
  • #108 — Let the current scene and the persistent map disagree about an open gate
Plausible inference substituted for direct evidence. Coordinate systems were not enforced architecturally. Diagnostics were added after failures instead of before policy. Completion was communicated too early or at the wrong layer.
— from "Patterns behind these"

A much shorter log covers the renderer: 27 mistakes from one shop-screen session.

Replay inspector

Watch the autopilot think

Record a run on the play page (the Record button, or Esc → RECORD), then replay it frame by frame through the same C autopilot, compiled to WebAssembly: its tactic, target, predicted paths and the controls it pressed, next to the game's own sprites.

replay/ · recordings stay in your browser

Offline route maps

Routes the autopilot plans over the complete level maps before play. Click through for the interactive versions.

Level 3 offline route

Level 3

 Level 4 offline route

Level 4

 Level 5 offline route

Level 5

Interactive explorers

Autopilot decision cost

Where each frame of autopilot thinking time goes, and how it scales.

autopilot-decision-cost.html

Space-time distance field

Two ways to ask "is this move safe": eager flood fill versus lazy search, measured.

hazard-distance-field-prototype.html

Level 2 navigation flow field

The offline flow field the autopilot follows through Level 2, phase by phase.

level-2-navigation-field.html

Level 2 route comparison

Candidate routes through Level 2 side by side.

level-2-route-comparison.html

Level 2 offline route

The planned route over the full Level 2 map.

level-2-offline-route.html

Level 3 offline route

The planned route over the full Level 3 map, including destructible gates.

level-3-offline-route.html

Level 4 offline route

The planned route over the full Level 4 map.

level-4-offline-route.html

Level 5 offline route

The planned route over the full Level 5 map, up to the tank boss.

level-5-offline-route.html

Start here

These notes are snapshots: each describes the controller as it was when written. The earlier ones cover the Python implementation; the native C port that followed changed a good deal of it, so prefer the more recent documents where they disagree.

The autopilot

The implementation guide: how it reads the game's memory, builds a world model, plans routes, picks tactics and drives the joystick. Start here.

AUTOPILOT.MD

228 ways an AI got it wrong

The full retrospective of every mistake the AI made while building the autopilot, most of them about walls, terrain and navigation, with root causes, the patterns behind them and the rules that came out of it.

AUTOPILOT_AIERRORS.MD

Finishing the game

Four uninterrupted campaigns from the first level to the destroyed Level 5 ship with no lives lost (2026-10-05), and the damage, missed pickups and rewards that still went wrong.

UNINTERRUPTED_CAMPAIGN_VALIDATION_20261005.MD

Why the ship kept getting stuck

The navigation review behind that result: where the ship stopped, a valid route was often already there, lost between planning and the joystick. Widening waypoints or forcing inputs did not fix it.

AUTOPILOT_NAVIGATION_REVIEW_20261005.MD

Level 2 postmortem

Why many individually plausible fixes did not make Level 2 reliable: the work was done in the wrong order.

LEVEL2_AUTOPILOT_POSTMORTEM.MD

The Level 5 tank

The tank boss's model, the many experiments that failed against it, and the C pilot's run that finally beat it without cheats (2026-09-29).

TANKS.MD

The Level 3 carrier

The articulated carrier boss: its tails, cores and health words, and the gun lane and refuges the autopilot uses against it.

LEVEL3_CARRIER_BOSS_TACTIC.MD

The Level 5 final ship

The final ship's damage sequence read from the original code, and the firing lanes that destroy its 18 mounts and collect all 20 cash drops.

LEVEL5_FINAL_BOSS_REVA.md

The replay inspector

How recordings are replayed in the browser through the same C autopilot compiled to WebAssembly, and what each overlay shows.

plan-web-replay-ui.md

Full-game regression recording

Recording complete playthroughs so that every change can be checked against a real run.

CAMPAIGN_VALIDATION.MD

The maneuver planner

The second-generation controller, its architecture, limits and how it compares to the original.

AUTOPILOT_MANEUVER.MD

Navigation optimization

The exact grid navigation backend and its live regression evidence.

AUTOPILOT_NAVIGATION.MD

Architecture review

Reducing decisions and queries per frame before porting the controller to C.

AUTOPILOT_ARCHITECTURE_REVIEW.MD

The native C port

Moving the controller from Python into dependency-free C inside the emulator, stage by stage.

AUTOPILOT_PERFORMANCE_C_PORT_PLAN.MD

Native session inside Hatari

The resident C session that drives the ship straight from the emulator's frame hook.

AUTOPILOT_NATIVE_SESSION.MD

Running the autopilot in the browser

Options for running the ~50k-line Python autopilot alongside the WASM game, including on phones.

AUTOPILOT_BROWSER_PORT_OPTIONS.MD

Level 2 strategy

Level 2 compiled into a static mission from the resident tile map.

LEVEL2_STRATEGY.MD

Level 3 strategy

A deliberately small mission policy for Level 3.

LEVEL3_STRATEGY.MD

Shop autopilot

Buying the right upgrades from Crispin using the game's own RAM state instead of screen reading.

plan-shop-autopilot.md
All autopilot notes (every tweak, kernel and validation run)

Working notes written during development. They are detailed, sometimes superseded by later ones, and kept for the record.