Luigi Pizzolito's Logo

Touhou 6 NEAT CNN - An Attempt was Made

Contents

Having recently grown rather fond of the Touhou game series, and spending some time with Touhou 6: Embodiment of Scarlet Devil; I thought to myself, how hard could it possibly be to train an AI to play this game better than my mere mortal fingers can?

Limited to one weekend’s worth of time before the beginning of my Master’s semester, I tried my best to get something working at all for this first attempt.

Introduction to Touhou

Touhou is a series of Japanese bullet hell shooter video games developed by Team Shanghai Alice; a.k.a. the solo developer ZUN. The games are known for their high difficulty, intricate patterns of enemy projectiles, and a large cast of characters. The series has a dedicated fanbase and has inspired a wide range of fan-made content, including music, art, and derivative games.

Touhou 6: Embodiment of Scarlet Devil

While it would be nice to hook into and use ZUN’s original 2002 game code directly, while some hooking libraries exist such as twinject, this would still be restrictive to the original game’s release which already has it’s own compatibility issues with modern versions of Windows and Direct X; often using community developed patches and wrappers such as thcrap.

To get around these limitations, I opted to look into the Touhou 6 Decompilation project, which aims to reverse engineer and decompile the original game code, making it more accessible for modern development and experimentation. Part of this effort involves a full, modern, SDL-based reimplementation of the game. Integration becomes straightforward as the source code is available and can be directly modified or extended for AI experimentation. As well as being cross-platform compatible, allowing me to work natively on Linux.

Network Input and Output Methods

The observation pipeline converts raw game state into a compact numeric vector that NEAT can evolve over. Two representation variants exist:

Raw Grid Input (Full 2050 inputs)

The traditional approach encodes the entire observation into a flat vector of 2050 inputs:

This yields 2050 raw inputs directly into NEAT’s genome. While expressive, the sparsity of meaningful signal relative to input dimensionality makes learning difficult.

Sparse 32×32 input grids The danger and powerup channels as rendered in the training UI

CNN-Preprocessed Input (66 inputs)

A convolutional autoencoder compresses the two 32×32 grids into a 64-dimensional latent vector before appending the player position:

                    ┌─────────────────────┐
                    │  Two 32×32 grids    │
                    │  (danger + powerup) │
                    └───────┬─────────────┘
                            ↓
                     GridAutoencoder
                            │
                     Encoder: Conv → 64 latent dims
                            │
        ┌──────────────────┼──────────────────┐
        ↓                  ↓                  ↓
CNN latent (64 dims)  Player X (norm)  Player Y (norm)
        ↓                  ↓                  ↓
        ────────────────────────────────────────
                             │
                      66 total inputs
                             │
                      → NEAT Genome
                             │
                      → 7 action outputs
                           (up/down/left/right/
                            shoot/bomb/focus)

The CNN autoencoder learns to reconstruct the observation grids from a latent space, preserving the most informative features while discarding sparse noise. Two variants were explored:

  1. Full-grid NEAT - feeds all 2050 raw inputs directly into NEAT evolution
  2. CNN-preprocessed NEAT - uses the autoencoder to reduce inputs to 66 dimensions (64 latent + 2 position values)

The CNN-preprocessed variant trains faster and achieved more stable results.

8×8 CNN latent representation The 64-dimensional latent vector visualized as an 8×8 heatmap in the CNN-NEAT UI

CNN autoencoder reconstruction The CNN autoencoder reconstruction quality, showing original grids (top row) with the decoded outputs (bottom row)

CNN Autoencoder Architecture & Training Data Pipeline

The convolutional autoencoder is the core preprocessing component that transforms sparse 32x32 observation grids into a compact 64-dimensional latent vector. Before diving into the architecture, it’s important to understand how the training data itself is collected.

How Training Data is Collected

During NEAT training, the ObservationEncoder (from encoder.py) creates player-centered raycast grids by rasterizing bullet paths, laser segments, enemies, and powerups into two 32x32 grids (danger and powerup). This process happens in real-time during gameplay:

  1. Grid generation: For each frame, the encoder converts the raw game state into:

    • A 32x32 danger grid representing bullet paths, laser segments, and enemy positions
    • A 32x32 powerup grid representing collectible items
  2. Chunked storage: The GridDataset class (from grid_dataset.py) accumulates these grid pairs in memory, saving them to disk as compressed .npz files when they reach the chunk size (default 25,000 samples per file).

  3. Dataset formation: These .npz files form the training dataset for the CNN autoencoder, which learns to reconstruct the grids from a compact latent representation.

The encoder adds each frame’s grids to the dataset via dataset.add(danger, powerups) during NEAT evaluation episodes. This creates a diverse dataset capturing various bullet patterns, powerup configurations, and game situations encountered during training.

Now, examining the autoencoder architecture in code/cnn_autoencoder.py:

cnn_autoencoder.py L113-L165
113class GridAutoencoder(nn.Module):
114    def __init__(self, latent_dim: int = 64):
115        super().__init__()
116
117        self.encoder = nn.Sequential(
118            nn.Conv2d(2, 16, 3, padding=1),
119            nn.ReLU(),
120
121            nn.Conv2d(16, 32, 3, stride=2, padding=1),
122            nn.ReLU(),
123
124            nn.Conv2d(32, 64, 3, stride=2, padding=1),
125            nn.ReLU(),
126
127            nn.Flatten(),
128
129            nn.Linear(64 * 8 * 8, latent_dim),
130        )
131
132        self.decoder = nn.Sequential(
133            nn.Linear(latent_dim, 64 * 8 * 8),
134            nn.ReLU(),
135
136            nn.Unflatten(1, (64, 8, 8)),
137
138            nn.ConvTranspose2d(
139                64,
140                32,
141                4,
142                stride=2,
143                padding=1,
144            ),
145            nn.ReLU(),
146
147            nn.ConvTranspose2d(
148                32,
149                16,
150                4,
151                stride=2,
152                padding=1,
153            ),
154            nn.ReLU(),
155
156            # Deliberately NO Sigmoid here.
157            #
158            # A Sigmoid allowed the decoder to collapse toward zero on
159            # the extremely sparse grids. Once saturated near zero,
160            # gradients through the Sigmoid became effectively zero.
161            nn.Conv2d(16, 2, 3, padding=1),
162        )
163
164    def forward(self, inputs: torch.Tensor) -> torch.Tensor:
165        return self.decoder(self.encoder(inputs))

The encoder consists of three convolutional layers that progressively reduce spatial dimensions while increasing feature depth:

  1. Conv2d(2, 16, 3, padding=1) - Input is the two-channel grid (danger + powerup). Output: 16 channels at 32x32 resolution. This layer detects low-level features such as individual bullet positions and powerup clusters.

  2. Conv2d(16, 32, 3, stride=2, padding=1) - Stride 2 halves the spatial dimensions to 16x16 while doubling channels to 32. This layer combines low-level features into mid-level patterns such as bullet trajectories and item groupings.

  3. Conv2d(32, 64, 3, stride=2, padding=1) - Stride 2 again reduces to 8x8 while doubling to 64 channels. This layer produces high-level abstractions: patterns indicating danger zones, powerup locations, and player-relative positioning.

  4. Flatten + Linear(6488, 64) - The 64x8x8 feature map (4,096 values) is flattened and projected down to 64 dimensions. This bottleneck forces the network to discard redundant information and retain only the most salient features.

The decoder mirrors the encoder using transposed convolutions to reconstruct the original 32x32 grids from the 64-dimensional latent vector. The decoder deliberately omits a final sigmoid activation because early experiments with sigmoid caused decoder collapse on sparse grids - once saturated near zero, gradients through the sigmoid became effectively zero, preventing learning.

The autoencoder is trained with a custom weighted MSE loss that gives 11x more gradient signal to active pixels (value 1.0) compared to empty pixels (value 0.0). Without this weighting, the decoder would learn to output all zeros since over 95% of the grid is empty space. The loss function is defined in the same file:

cnn_autoencoder.py L168-L186
168def weighted_mse(
169    reconstruction: torch.Tensor,
170    target: torch.Tensor,
171) -> torch.Tensor:
172    """
173    MSE with increased weight on active pixels.
174
175    Empty pixels:
176        weight = 1
177
178    Target = 0.5:
179        weight = 6
180
181    Target = 1:
182        weight = 11
183    """
184    weight = 1.0 + 10.0 * target
185    return ((reconstruction - target).square() * weight).mean()

The weighting scheme gives 11× more gradient signal to fully active pixels (value 1.0) compared to empty pixels (value 0.0). Without this, the decoder would simply learn to output all zeros since >95% of the grid is empty space, and the gradients from the few active pixels would be drowned out.

Training proceeds through standard epochs with checkpoint handling. A key detail is that old checkpoints trained with a sigmoid output and plain MSE are intentionally ignored, forcing a fresh start with the corrected architecture:

cnn_autoencoder.py L201-L320
201def train(
202    dataset_directory: Path,
203    output_path: Path,
204    latent_dim: int = 64,
205    batch_size: int = 256,
206    epochs: int = 30,
207) -> None:
208    files = sorted(dataset_directory.glob("chunk_*.npz"))
209
210    if not files:
211        raise RuntimeError(
212            f"No dataset chunks found in {dataset_directory}"
213        )
214
215    dataset = GridBatchDataset(
216        files,
217        batch_size,
218        shuffle=True,
219    )
220
221    device = training_device()
222
223    loader = DataLoader(
224        dataset,
225        batch_size=None,
226        num_workers=0,
227        pin_memory=device.type in {"cuda", "xpu"},
228    )
229
230    checkpoint = None
231    completed_epochs = 0
232
233    # ------------------------------------------------------------------
234    # Checkpoint handling
235    #
236    # The old checkpoint was trained with a Sigmoid output and plain MSE.
237    # Do NOT resume it: start a fresh model with the fixed architecture.
238    # ------------------------------------------------------------------
239
240    if output_path.exists():
241        candidate = torch.load(
242            output_path,
243            map_location=device,
244            weights_only=False,
245        )
246
247        if (
248            isinstance(candidate, dict)
249            and candidate.get("checkpoint_version") == CHECKPOINT_VERSION
250            and "model" in candidate
251        ):
252            checkpoint = candidate
253            latent_dim = checkpoint.get("latent_dim", latent_dim)
254            completed_epochs = checkpoint.get("completed_epochs", 0)
255
256            print(
257                f"Resuming checkpoint at epoch "
258                f"{completed_epochs:,} from {output_path}."
259            )
260        else:
261            print(
262                f"Ignoring incompatible old checkpoint at {output_path}; "
263                "starting fresh with the fixed autoencoder."
264            )
265
266    model = GridAutoencoder(
267        latent_dim=latent_dim,
268    ).to(device)
269
270    optimizer = torch.optim.AdamW(
271        model.parameters(),
272        lr=1e-3,
273        weight_decay=1e-5,
274    )
275
276    if checkpoint is not None:
277        model.load_state_dict(checkpoint["model"])
278
279        if "optimizer" in checkpoint:
280            optimizer.load_state_dict(checkpoint["optimizer"])
281
282    print(
283        f"Training on {dataset.sample_count:,} non-empty samples "
284        f"from {len(files)} chunks using {device}; "
285        f"removed {dataset.empty_samples:,} empty samples."
286    )
287
288    print(
289        f"latent_dim={latent_dim}, "
290        f"batch_size={batch_size}, "
291        f"loss=weighted_mse"
292    )
293
294    output_path.parent.mkdir(
295        parents=True,
296        exist_ok=True,
297    )
298
299    for epoch in range(
300        completed_epochs,
301        completed_epochs + epochs,
302    ):
303        model.train()
304
305        total_loss = 0.0
306        processed_samples = 0
307
308        for inputs in loader:
309            inputs = inputs.to(
310                device,
311                non_blocking=True,
312            )
313
314            # Forward pass.
315            reconstruction = model(inputs)
316
317            # Weighted reconstruction loss.
318            loss = weighted_mse(
319                reconstruction,
320                inputs,

Evaluating Reconstruction Quality

Evaluating the reconstruction is critical before using the autoencoder for NEAT training. A low training loss alone is not enough, because the model could simply learn to output all zeros or memorize common patterns while failing to preserve the rare but gameplay-critical features. The training loop addresses this by logging reconstruction statistics at each epoch:

cnn_autoencoder.py L350-L410
350        # Inspect the final decoder output distribution.
351        model.eval()
352
353        with torch.no_grad():
354            sample_inputs = next(iter(loader)).to(
355                device,
356                non_blocking=True,
357            )
358
359            sample_reconstruction = model(sample_inputs)
360
361            reconstruction_mean = (
362                sample_reconstruction.mean().item()
363            )
364
365            reconstruction_max = (
366                sample_reconstruction.max().item()
367            )
368
369            reconstruction_min = (
370                sample_reconstruction.min().item()
371            )
372
373            target_mean = sample_inputs.mean().item()
374
375            target_active_fraction = (
376                (sample_inputs > 0.01)
377                .float()
378                .mean()
379                .item()
380            )
381
382            reconstruction_active_fraction = (
383                (sample_reconstruction > 0.01)
384                .float()
385                .mean()
386                .item()
387            )
388
389        print(
390            f"epoch {epoch + 1}: "
391            f"loss={epoch_loss:.12f}"
392        )
393
394        print(
395            f"  params={param_norm:.6f} "
396            f"grads={grad_norm:.6f}"
397        )
398
399        print(
400            f"  target_mean={target_mean:.6f} "
401            f"target_active={target_active_fraction:.4%}"
402        )
403
404        print(
405            f"  reconstruction="
406            f"[{reconstruction_min:.4f}, "
407            f"{reconstruction_max:.4f}] "
408            f"mean={reconstruction_mean:.6f} "
409            f"active={reconstruction_active_fraction:.4%}"
410        )

These diagnostics track:

The repository also includes an interactive reconstruction viewer (python/cnn/view_autoencoder.py) that randomly samples grid pairs, runs them through the autoencoder, and displays the original and reconstructed grids side by side. This visual inspection is essential because numerical metrics alone cannot reveal whether the model has preserved spatial relationships like bullet trajectories, laser shapes, or powerup paths. For example, a model with good overall MSE might still blur bullet positions into the wrong grid cells, which would be catastrophic for a controller that needs precise dodging.

The reconstruction image in this section shows a representative sample from this visual evaluation process. If the reconstructed grids fail to preserve the spatial arrangement of the original grids, the latent vector is not suitable for downstream NEAT training.

Why the CNN Input Was Necessary

The raw 2050-dimensional observation created a credit-assignment nightmare for NEAT. In a typical frame, over 95% of the danger grid cells are zero, meaning most genome weights receive no meaningful gradient signal. The CNN bottleneck compresses the most salient features into 66 dimensions (64 latent + 2 position), reducing the search space by 30x and eliminating the noise from empty grid cells.

However, this compression introduces a new limitation: the 64-dimensional latent vector is a flat representation with no preserved spatial structure. Although the encoder produces a 64x8x8 feature map internally, the final Linear layer mixes all spatial positions together before outputting the latent vector. The NEAT controller therefore receives 64 independent values that must be interpreted without explicit spatial layout, making it difficult to learn spatial relationships such as “bullets are approaching from the right side” directly from the latent representation.

The CNN-specific training UI visualizes the latent vector as an 8x8 heatmap for manual inspection, but this is purely a display convenience. The actual input to the NEAT genome is the flat 64-element vector plus two position values, totaling 66 inputs.

Training & Simulation Framework

The training pipeline orchestrates three main components:

  1. Game environment - TH06 running headless via the decompilation port, exposed over a Unix socket
  2. Observation encoder - converts game state into either raw 32×32 grids or CNN-latent vectors
  3. NEAT evolution loop - neuroevolutionary search over neural network controllers

The CnnNeatTrainer class manages the full training lifecycle:

The trainer factory methods that wire up the CNN encoder and UI:

cnn_trainer.py L101-L113
101def cnn_encoder_factory(config):
102    if config.dataset_directory is not None:
103        config.dataset_directory = PROJECT_DIRECTORY / "dataset"
104    encoder = CnnObservationEncoder(config, WEIGHTS_PATH, use_cpu=force_cpu)
105    encoder_holder["encoder"] = encoder
106    return encoder
107
108
109def cnn_ui_factory(*args, **kwargs):
110    ui = CnnTrainingUi(*args, **kwargs)
111    encoder_holder["encoder"].set_ui(ui)
112    return ui

The observation encoder transforms raw game state into the appropriate input format. The CnnObservationEncoder from code/cnn_encoder.py performs this transformation:

cnn_encoder.py L39-L61
39    @property
40    def dim(self) -> int:
41        return self.latent_dim + 2
42
43    def set_ui(self, ui: CnnTrainingUi) -> None:
44        self.ui = ui
45
46    def reset(self) -> None:
47        self.raw_encoder.reset()
48
49    def flush_dataset(self) -> None:
50        self.raw_encoder.flush_dataset()
51
52    def encode(self, state: StateMessage, previous_action: int = 0) -> list[float]:
53        raw_observation = self.raw_encoder.encode(state, previous_action)
54        raw_grids = np.asarray(raw_observation[:-2], dtype=np.float32).reshape(1, 2, 32, 32)
55        inputs = torch.from_numpy(np.clip(raw_grids, 0.0, 1.0)).to(self.device)
56        with torch.inference_mode():
57            latent = self.model.encoder(inputs)[0].cpu().tolist()
58        if self.ui is not None:
59            self.ui.set_raw_observation(raw_observation[:-2])
60            self.ui.set_cnn_latent(latent)
61        return latent + raw_observation[-2:]

The CNN-specific UI panel renders the raw observation grids and the latent heatmap:

cnn_training_ui.py L44-L65
44    def draw(self, generation: int, best_genome=None, current_genome: int | None = None) -> None:
45        super().draw(generation + self.generation_offset, best_genome, current_genome)
46
47    def _observation_panel(self, rect: pygame.Rect) -> None:
48        pygame.draw.rect(self.screen, (24, 30, 45), rect)
49        self._text("CNN INPUT / LATENT", (rect.x + 10, rect.y + 10), (225, 232, 245), self.small_font)
50        if self.raw_observation is None:
51            self._text("waiting for observation", (rect.x + 10, rect.y + 34), (130, 145, 165), self.small_font)
52            return
53        size = min(280, (rect.width - 68) // 2)
54        gap = 32
55        start_x = rect.x + (rect.width - 2 * size - gap) // 2
56        for layer, (label, color) in enumerate((("danger path", (255, 75, 65)), ("powerups", (85, 170, 255)))):
57            x = start_x + layer * (size + gap)
58            self._text(label, (x, rect.y + 20), color, self.small_font)
59            self._grid(pygame.Rect(x, rect.y + 42, size, size), self.raw_observation[layer * 1024:(layer + 1) * 1024], color)
60        self._text(f"grid samples collected: {self.dataset_samples:,}", (start_x, rect.y + 330),
61                   (180, 195, 220), self.small_font)
62        self._text("CNN latent (8x8)", (start_x, rect.y + 350), (170, 190, 255), self.small_font)
63        if self.cnn_latent is not None:
64            self._latent_grid(pygame.Rect(start_x + 140, rect.y + 342, 80, 80), self.cnn_latent)

Baseline Controllers

Before NEAT evolution, several baseline controllers were implemented to establish performance benchmarks. These are defined in code/controllers.py:

controllers.py L11-L51
11class NoOpController:
12    """Controller that takes no action (all inputs 0)."""
13
14    def act(self, state: StateMessage) -> int:
15        return 0
16
17
18class FixedCenterController:
19    """
20    Controller that moves the player to the center of the screen.
21
22    Reads current player position and issues input to move towards center.
23    """
24
25    def __init__(self, center_x: float = 192.0, center_y: float = 224.0):
26        self.center_x = center_x
27        self.center_y = center_y
28
29    def act(self, state: StateMessage) -> int:
30        dx = self.center_x - state.player_x
31        dy = self.center_y - state.player_y
32
33        action = 0
34        if abs(dx) > 5:
35            action |= 0x02 if dx > 0 else 0x01  # right = 2, left = 1
36        if abs(dy) > 5:
37            action |= 0x08 if dy > 0 else 0x04  # down = 8, up = 4
38        return action
39
40
41class RandomController:
42    """Controller that selects a random action (for baseline)."""
43
44    def __init__(self, seed: int | None = None):
45        import random
46
47        self.random = random.Random(seed)
48
49    def act(self, state: StateMessage) -> int:
50        return self.random.randint(0, 255)

These controllers served as negative controls and helped diagnose whether the NEAT-evolved genomes were actually learning meaningful behavior.

Reward Function Design & Iterative Tuning

A key aspect of this project was the iterative refinement of the reward function across multiple development phases. The reward components in th06_env.py show clear phase-based evolution, reflecting an ongoing process of fine-tuning based on empirical results. The RewardComponents class tracks each term separately:

1class RewardComponents:
2    """Tracks individual reward components for transparent logging."""
3
4    __slots__ = ("survival", "score_delta", "graze_delta", "powerup", "focus_danger",
5                 "bomb_misuse", "edge_hugging", "death_penalty", "edge_termination", "total")

The final configuration after three phases of tuning:

 1# Survival reward (always constant)
 2SURVIVAL_REWARD: float = 0.01
 3
 4# Score-based rewards
 5SCORE_REWARD_DIVISOR: float = 1000000.0
 6GRAZE_REWARD: float = 100.0
 7POWERUP_REWARD: float = 350.0
 8FOCUS_DANGER_REWARD: float = 5.0
 9FOCUS_DANGER_RADIUS_MULTIPLIER: float = 8.0
10
11# Penalty terms (showing phase evolution)
12BOMB_MISUSE_PENALTY: float = 350.0     # Phase 2: 150.0    # Phase 1: 50.0
13BOMB_DANGER_BULLET_THRESHOLD: int = 3
14
15EDGE_HUGGING_PENALTY: float = 100.0    # Phase 3: 10.0     # Phase 2: 5.0      # Phase 1: 2.0
16EDGE_HUGGING_MARGIN: float = 48.0      # Phase 2: 24.0
17EDGE_HUGGING_THRESHOLD: int = 6000     # Phase 2: 6000000000
18EDGE_HUGGING_TERMINATION_PENALTY: float = 5000.0
19
20DEATH_PENALTY: float = 25000.0
21MAX_FRAMES_PER_EPISODE: int = 12000    # Phase 2: 6000

Phase 1 (Initial Experiments)

Extremely lenient penalties: Bomb misuse at -50, edge hugging at -2/frame. Very short episodes at 6,000 frames maximum. The goal was simply to establish the basic framework and verify NEAT could learn anything at all. At this stage, the genomes were dying almost immediately from bomb spam or walking into corners, but the signal was too weak to discourage these behaviors.

Phase 2 (Stabilization)

Moderate penalties: Bomb misuse increased to -150, edge hugging to -5/frame. Episode length doubled to 12,000 frames maximum. Edge hugging margin reduced from effectively infinite to 48 pixels (roughly 1/8th of the playfield width). The goal was to discourage obvious maladaptive behaviors while still rewarding exploration. This helped, but genomes still tended to bomb too early or hug edges when under pressure.

Phase 3 (Fine-tuning)

Strong penalties: Bomb misuse at -350, edge hugging at -100/frame. Continued long episodes with refined edge detection. The goal was to create a clear survival signal where even small mistakes are heavily penalized. The edge hugging threshold of 6000 frames means if the player stays within 48 pixels of any screen edge for that many consecutive frames (about 100 seconds at 60 FPS), the episode terminates with a -5000 penalty.

The bomb misuse logic checks if the player presses bomb when fewer than 3 bullets are within the focus danger radius (player hitbox 1.25 × 8.0 = 10 pixels). This prevents wasting bombs on sparse patterns while still allowing them for dense waves.

The focus danger reward gives +5 per frame when holding focus (shift) and at least one bullet is within the danger radius. This encourages the AI to use focused movement for precision dodging rather than just bombing.

The death penalty at -25,000 is deliberately massive compared to any positive reward. A single death wipes out 250,000 frames of survival reward, or roughly 70 powerup collections. This forces the NEAT population to prioritize survival above all else.

This evolution reflects the fundamental challenge in sparse-reward environments like Touhou 6: the agent receives meaningful feedback only for rare events (collecting powerups: +350, grazing bullets: +100) or catastrophic events (death: -25,000). Most frames provide only the minimal survival reward (+0.01), making credit assignment difficult.

The NEAT training configuration for the CNN-preprocessed variant reflects these insights, using a population size of 200 to maintain diversity while searching for genomes that can navigate this sparse reward landscape.

Results

Despite experimentation with both full-grid and CNN-preprocessed NEAT, the AI was unable to play Touhou 6 above random performance levels. The final generation (generation 49) of the CNN-preprocessed run produced the following metrics:

Metric Value
Best fitness -16,142
Average fitness -171,165
Best score 1,256,350
Average score 214,733
Best survival frames ~2,599
Best powerups collected 23
Best grazes 6
Episodes completed 0

The best genome survived approximately 2,599 frames (25.99 survival reward / 0.01 per frame), collected 23 powerups (8,050 reward / 350 each), and grazed 6 bullets (600 reward / 100 each). However, it still died, incurring the full -25,000 death penalty.

Across all 150 evaluated genomes, the average reward components show the core problems:

Reward Component Average Value
Survival +20.98
Graze -3.33
Powerups +7,170
Focus danger +134
Bomb misuse -71,888
Edge hugging -81,600
Death penalty -25,000

The bomb misuse and edge hugging penalties dominated the average reward, showing that the evolved genomes were repeatedly bombing too early and hiding near screen edges. Every genome died, which is why the best fitness remained negative despite the positive rewards from powerups, graze, and focus.

The CNN preprocessing did improve consistency compared to the full-grid variant, but the reward signal proved too sparse for effective credit assignment within the limited training budget (one weekend, 50 generations, population 200).

Baseline controllers performed as expected:

The full training logs and per-generation statistics are available in the neat_output/ directory of the repository.

Reproducing the Project

If you want to run this project yourself or build on it, here are the practical setup details from the repository:

Game Data (Required)

The decompilation project requires assets from a legally owned copy of Touhou 6: Embodiment of Scarlet Devil. Copy these into the repository root:

The project cannot run without these files - the C++ binary loads them at startup.

C++ Build Process

The game engine is built from the modified decompilation source using Premake5 and Make:

1# Install system dependencies (Debian/Ubuntu)
2sudo apt install build-essential libsdl2-dev libsdl2-image-dev libsdl2-ttf-dev libasound2-dev
3
4# Generate build files and compile
5premake5 gmake && cd build && make -j16 && cd ..

The th06 binary will appear in the repository root. This binary includes the AI input socket, headless mode, and frameskip acceleration needed for training.

Python Dependencies

1python/.venv/bin/python -m pip install -r python/requirements.txt

Key dependencies: NEAT-Python, PyTorch, pygame, numpy.

Training Pipeline

The full pipeline has four stages:

  1. Collect dataset - Run full-grid NEAT with dataset saving to generate dataset/chunk_*.npz files:

    1python/.venv/bin/python launch_instances.py --neat --frameskip 10000 --no-brain --quiet
    
  2. Train CNN autoencoder - Compress the 32×32 grids to 64-dim latent vectors:

    1cd python/cnn
    2../.venv/bin/python trainer.py
    

    Weights saved to python/cnn/weights/grid_autoencoder.pt.

  3. Inspect reconstructions - Verify the autoencoder preserves spatial details:

    1cd python/cnn
    2../.venv/bin/python view_autoencoder.py
    

    Keyboard: Right/Space next, Left previous, R random, E test empty sample, Q quit.

  4. Train CNN-NEAT - Headless training with live gnuplot reward plots:

    1cd python/cnn-train
    2../.venv/bin/python launch_instances.py --neat --frameskip 10000 --quiet --no-dataset-save --cpu --no-ui
    

    In a separate terminal:

    1gnuplot -persist plot.gp
    

For a full-grid NEAT baseline (without CNN), omit the --cpu flag and run from the main python directory.

Why the Project Failed

The project failed for two interconnected reasons:

  1. Excessive corner-hugging behavior: Despite the edge hugging penalty, the AI learned that staying in corners minimized immediate danger from bullets. The edge detection system penalized proximity to edges but did not distinguish between active dodging (moving along edges to avoid patterns) and passive hiding (remaining stationary in corners). Over long episodes, the small survival reward (+0.01 per frame) accumulated to outweigh the periodic edge hugging penalties, especially when the AI could survive tens of thousands of frames in a corner before penalties became significant.

  2. Insufficient spatial detail in CNN grids: The 32×32 input grids lacked the resolution needed for precise bullet dodging. Each grid cell represented approximately 12×12 pixels of the actual game screen. This coarse granularity made it impossible for the CNN to learn fine-grained patterns of bullet trajectories. The autoencoder’s latent space, while compressing the most active features, could not recover the spatial precision lost in the initial downsampling to 32×32. Consequently, the NEAT controller received observation vectors that lacked the detail necessary to distinguish between safe and unsafe micro-movements within a grid cell.

These two factors combined to create a local optimum where corner hugging with occasional focusing emerged as the highest-reward strategy, but the AI was inadequate for actual Touhou 6 gameplay which requires precise movement through complex bullet patterns.

Next Steps

The attempt, while unsuccessful in achieving playable AI, revealed several insights:

  1. CNN preprocessing aids learning - reducing 2050 inputs to 66 latent + position values made training more stable and faster
  2. Reward sparsity is the primary bottleneck - the game provides meaningful rewards only for graze (100), powerup collection (350), and death events; intermediate gameplay yields minimal signal
  3. Population size matters - 200 genomes provided diversity but insufficient for the credit-assignment problem within limited training time
  4. Full-grid NEAT struggles more - the raw 2050-dimensional observation creates a credit-assignment nightmare; the CNN bottleneck actually helps
  5. Iterative reward shaping is essential - the phased evolution of penalty terms (particularly for bomb misuse and edge hugging) was critical for making any progress at all
  6. Spatial resolution is critical for bullet dodging - the 32×32 grid representation loses too much precision for pattern recognition
  7. Latent space interpretability matters - the 64-dimensional latent vector, while computationally efficient, obscures the spatial relationships NEAT needs to evolve effective dodging strategies

Future directions include:

The project remains a valuable exploration of the intersection between classical neuroevolution and modern deep learning preprocessing for classic game AI.

Categories
Tags