Touhou 6 NEAT CNN - An Attempt was Made
Contents
Having recently grown rather fond of the Touhou game series, and spending some time with Touhou 6: Embodiment of Scarlet Devil; I thought to myself, how hard could it possibly be to train an AI to play this game better than my mere mortal fingers can?
Limited to one weekend’s worth of time before the beginning of my Master’s semester, I tried my best to get something working at all for this first attempt.
Introduction to Touhou
Touhou is a series of Japanese bullet hell shooter video games developed by Team Shanghai Alice; a.k.a. the solo developer ZUN. The games are known for their high difficulty, intricate patterns of enemy projectiles, and a large cast of characters. The series has a dedicated fanbase and has inspired a wide range of fan-made content, including music, art, and derivative games.
While it would be nice to hook into and use ZUN’s original 2002 game code directly, while some hooking libraries exist such as twinject, this would still be restrictive to the original game’s release which already has it’s own compatibility issues with modern versions of Windows and Direct X; often using community developed patches and wrappers such as thcrap.
To get around these limitations, I opted to look into the Touhou 6 Decompilation project, which aims to reverse engineer and decompile the original game code, making it more accessible for modern development and experimentation. Part of this effort involves a full, modern, SDL-based reimplementation of the game. Integration becomes straightforward as the source code is available and can be directly modified or extended for AI experimentation. As well as being cross-platform compatible, allowing me to work natively on Linux.
Network Input and Output Methods
The observation pipeline converts raw game state into a compact numeric vector that NEAT can evolve over. Two representation variants exist:
Raw Grid Input (Full 2050 inputs)
The traditional approach encodes the entire observation into a flat vector of 2050 inputs:
- 32×32 danger grid (bullet paths, laser segments, enemy positions) → 1024 values
- 32×32 powerup grid (collectible items) → 1024 values
- Player X position normalized → 1 value
- Player Y position normalized → 1 value
This yields 2050 raw inputs directly into NEAT’s genome. While expressive, the sparsity of meaningful signal relative to input dimensionality makes learning difficult.
The danger and powerup channels as rendered in the training UI
CNN-Preprocessed Input (66 inputs)
A convolutional autoencoder compresses the two 32×32 grids into a 64-dimensional latent vector before appending the player position:
┌─────────────────────┐
│ Two 32×32 grids │
│ (danger + powerup) │
└───────┬─────────────┘
↓
GridAutoencoder
│
Encoder: Conv → 64 latent dims
│
┌──────────────────┼──────────────────┐
↓ ↓ ↓
CNN latent (64 dims) Player X (norm) Player Y (norm)
↓ ↓ ↓
────────────────────────────────────────
│
66 total inputs
│
→ NEAT Genome
│
→ 7 action outputs
(up/down/left/right/
shoot/bomb/focus)
The CNN autoencoder learns to reconstruct the observation grids from a latent space, preserving the most informative features while discarding sparse noise. Two variants were explored:
- Full-grid NEAT - feeds all 2050 raw inputs directly into NEAT evolution
- CNN-preprocessed NEAT - uses the autoencoder to reduce inputs to 66 dimensions (64 latent + 2 position values)
The CNN-preprocessed variant trains faster and achieved more stable results.
The 64-dimensional latent vector visualized as an 8×8 heatmap in the CNN-NEAT UI
The CNN autoencoder reconstruction quality, showing original grids (top row) with the decoded outputs (bottom row)
CNN Autoencoder Architecture & Training Data Pipeline
The convolutional autoencoder is the core preprocessing component that transforms sparse 32x32 observation grids into a compact 64-dimensional latent vector. Before diving into the architecture, it’s important to understand how the training data itself is collected.
How Training Data is Collected
During NEAT training, the ObservationEncoder (from encoder.py) creates player-centered raycast grids by rasterizing bullet paths, laser segments, enemies, and powerups into two 32x32 grids (danger and powerup). This process happens in real-time during gameplay:
-
Grid generation: For each frame, the encoder converts the raw game state into:
- A 32x32 danger grid representing bullet paths, laser segments, and enemy positions
- A 32x32 powerup grid representing collectible items
-
Chunked storage: The
GridDatasetclass (fromgrid_dataset.py) accumulates these grid pairs in memory, saving them to disk as compressed.npzfiles when they reach the chunk size (default 25,000 samples per file). -
Dataset formation: These
.npzfiles form the training dataset for the CNN autoencoder, which learns to reconstruct the grids from a compact latent representation.
The encoder adds each frame’s grids to the dataset via dataset.add(danger, powerups) during NEAT evaluation episodes. This creates a diverse dataset capturing various bullet patterns, powerup configurations, and game situations encountered during training.
Now, examining the autoencoder architecture in code/cnn_autoencoder.py:
113class GridAutoencoder(nn.Module):
114 def __init__(self, latent_dim: int = 64):
115 super().__init__()
116
117 self.encoder = nn.Sequential(
118 nn.Conv2d(2, 16, 3, padding=1),
119 nn.ReLU(),
120
121 nn.Conv2d(16, 32, 3, stride=2, padding=1),
122 nn.ReLU(),
123
124 nn.Conv2d(32, 64, 3, stride=2, padding=1),
125 nn.ReLU(),
126
127 nn.Flatten(),
128
129 nn.Linear(64 * 8 * 8, latent_dim),
130 )
131
132 self.decoder = nn.Sequential(
133 nn.Linear(latent_dim, 64 * 8 * 8),
134 nn.ReLU(),
135
136 nn.Unflatten(1, (64, 8, 8)),
137
138 nn.ConvTranspose2d(
139 64,
140 32,
141 4,
142 stride=2,
143 padding=1,
144 ),
145 nn.ReLU(),
146
147 nn.ConvTranspose2d(
148 32,
149 16,
150 4,
151 stride=2,
152 padding=1,
153 ),
154 nn.ReLU(),
155
156 # Deliberately NO Sigmoid here.
157 #
158 # A Sigmoid allowed the decoder to collapse toward zero on
159 # the extremely sparse grids. Once saturated near zero,
160 # gradients through the Sigmoid became effectively zero.
161 nn.Conv2d(16, 2, 3, padding=1),
162 )
163
164 def forward(self, inputs: torch.Tensor) -> torch.Tensor:
165 return self.decoder(self.encoder(inputs))The encoder consists of three convolutional layers that progressively reduce spatial dimensions while increasing feature depth:
-
Conv2d(2, 16, 3, padding=1) - Input is the two-channel grid (danger + powerup). Output: 16 channels at 32x32 resolution. This layer detects low-level features such as individual bullet positions and powerup clusters.
-
Conv2d(16, 32, 3, stride=2, padding=1) - Stride 2 halves the spatial dimensions to 16x16 while doubling channels to 32. This layer combines low-level features into mid-level patterns such as bullet trajectories and item groupings.
-
Conv2d(32, 64, 3, stride=2, padding=1) - Stride 2 again reduces to 8x8 while doubling to 64 channels. This layer produces high-level abstractions: patterns indicating danger zones, powerup locations, and player-relative positioning.
-
Flatten + Linear(6488, 64) - The 64x8x8 feature map (4,096 values) is flattened and projected down to 64 dimensions. This bottleneck forces the network to discard redundant information and retain only the most salient features.
The decoder mirrors the encoder using transposed convolutions to reconstruct the original 32x32 grids from the 64-dimensional latent vector. The decoder deliberately omits a final sigmoid activation because early experiments with sigmoid caused decoder collapse on sparse grids - once saturated near zero, gradients through the sigmoid became effectively zero, preventing learning.
The autoencoder is trained with a custom weighted MSE loss that gives 11x more gradient signal to active pixels (value 1.0) compared to empty pixels (value 0.0). Without this weighting, the decoder would learn to output all zeros since over 95% of the grid is empty space. The loss function is defined in the same file:
168def weighted_mse(
169 reconstruction: torch.Tensor,
170 target: torch.Tensor,
171) -> torch.Tensor:
172 """
173 MSE with increased weight on active pixels.
174
175 Empty pixels:
176 weight = 1
177
178 Target = 0.5:
179 weight = 6
180
181 Target = 1:
182 weight = 11
183 """
184 weight = 1.0 + 10.0 * target
185 return ((reconstruction - target).square() * weight).mean()
The weighting scheme gives 11× more gradient signal to fully active pixels (value 1.0) compared to empty pixels (value 0.0). Without this, the decoder would simply learn to output all zeros since >95% of the grid is empty space, and the gradients from the few active pixels would be drowned out.
Training proceeds through standard epochs with checkpoint handling. A key detail is that old checkpoints trained with a sigmoid output and plain MSE are intentionally ignored, forcing a fresh start with the corrected architecture:
201def train(
202 dataset_directory: Path,
203 output_path: Path,
204 latent_dim: int = 64,
205 batch_size: int = 256,
206 epochs: int = 30,
207) -> None:
208 files = sorted(dataset_directory.glob("chunk_*.npz"))
209
210 if not files:
211 raise RuntimeError(
212 f"No dataset chunks found in {dataset_directory}"
213 )
214
215 dataset = GridBatchDataset(
216 files,
217 batch_size,
218 shuffle=True,
219 )
220
221 device = training_device()
222
223 loader = DataLoader(
224 dataset,
225 batch_size=None,
226 num_workers=0,
227 pin_memory=device.type in {"cuda", "xpu"},
228 )
229
230 checkpoint = None
231 completed_epochs = 0
232
233 # ------------------------------------------------------------------
234 # Checkpoint handling
235 #
236 # The old checkpoint was trained with a Sigmoid output and plain MSE.
237 # Do NOT resume it: start a fresh model with the fixed architecture.
238 # ------------------------------------------------------------------
239
240 if output_path.exists():
241 candidate = torch.load(
242 output_path,
243 map_location=device,
244 weights_only=False,
245 )
246
247 if (
248 isinstance(candidate, dict)
249 and candidate.get("checkpoint_version") == CHECKPOINT_VERSION
250 and "model" in candidate
251 ):
252 checkpoint = candidate
253 latent_dim = checkpoint.get("latent_dim", latent_dim)
254 completed_epochs = checkpoint.get("completed_epochs", 0)
255
256 print(
257 f"Resuming checkpoint at epoch "
258 f"{completed_epochs:,} from {output_path}."
259 )
260 else:
261 print(
262 f"Ignoring incompatible old checkpoint at {output_path}; "
263 "starting fresh with the fixed autoencoder."
264 )
265
266 model = GridAutoencoder(
267 latent_dim=latent_dim,
268 ).to(device)
269
270 optimizer = torch.optim.AdamW(
271 model.parameters(),
272 lr=1e-3,
273 weight_decay=1e-5,
274 )
275
276 if checkpoint is not None:
277 model.load_state_dict(checkpoint["model"])
278
279 if "optimizer" in checkpoint:
280 optimizer.load_state_dict(checkpoint["optimizer"])
281
282 print(
283 f"Training on {dataset.sample_count:,} non-empty samples "
284 f"from {len(files)} chunks using {device}; "
285 f"removed {dataset.empty_samples:,} empty samples."
286 )
287
288 print(
289 f"latent_dim={latent_dim}, "
290 f"batch_size={batch_size}, "
291 f"loss=weighted_mse"
292 )
293
294 output_path.parent.mkdir(
295 parents=True,
296 exist_ok=True,
297 )
298
299 for epoch in range(
300 completed_epochs,
301 completed_epochs + epochs,
302 ):
303 model.train()
304
305 total_loss = 0.0
306 processed_samples = 0
307
308 for inputs in loader:
309 inputs = inputs.to(
310 device,
311 non_blocking=True,
312 )
313
314 # Forward pass.
315 reconstruction = model(inputs)
316
317 # Weighted reconstruction loss.
318 loss = weighted_mse(
319 reconstruction,
320 inputs,Evaluating Reconstruction Quality
Evaluating the reconstruction is critical before using the autoencoder for NEAT training. A low training loss alone is not enough, because the model could simply learn to output all zeros or memorize common patterns while failing to preserve the rare but gameplay-critical features. The training loop addresses this by logging reconstruction statistics at each epoch:
350 # Inspect the final decoder output distribution.
351 model.eval()
352
353 with torch.no_grad():
354 sample_inputs = next(iter(loader)).to(
355 device,
356 non_blocking=True,
357 )
358
359 sample_reconstruction = model(sample_inputs)
360
361 reconstruction_mean = (
362 sample_reconstruction.mean().item()
363 )
364
365 reconstruction_max = (
366 sample_reconstruction.max().item()
367 )
368
369 reconstruction_min = (
370 sample_reconstruction.min().item()
371 )
372
373 target_mean = sample_inputs.mean().item()
374
375 target_active_fraction = (
376 (sample_inputs > 0.01)
377 .float()
378 .mean()
379 .item()
380 )
381
382 reconstruction_active_fraction = (
383 (sample_reconstruction > 0.01)
384 .float()
385 .mean()
386 .item()
387 )
388
389 print(
390 f"epoch {epoch + 1}: "
391 f"loss={epoch_loss:.12f}"
392 )
393
394 print(
395 f" params={param_norm:.6f} "
396 f"grads={grad_norm:.6f}"
397 )
398
399 print(
400 f" target_mean={target_mean:.6f} "
401 f"target_active={target_active_fraction:.4%}"
402 )
403
404 print(
405 f" reconstruction="
406 f"[{reconstruction_min:.4f}, "
407 f"{reconstruction_max:.4f}] "
408 f"mean={reconstruction_mean:.6f} "
409 f"active={reconstruction_active_fraction:.4%}"
410 )These diagnostics track:
- Overall MSE: The average reconstruction error across all pixels
- Active MSE: Reconstruction error only on non-zero pixels, which is the more important metric given the extreme sparsity
- Target vs reconstruction statistics: Mean, min, and max values that reveal whether the decoder is saturating or collapsing
- Active pixel fractions: How many pixels the model marks as occupied, which reveals mode collapse or overfitting
The repository also includes an interactive reconstruction viewer (python/cnn/view_autoencoder.py) that randomly samples grid pairs, runs them through the autoencoder, and displays the original and reconstructed grids side by side. This visual inspection is essential because numerical metrics alone cannot reveal whether the model has preserved spatial relationships like bullet trajectories, laser shapes, or powerup paths. For example, a model with good overall MSE might still blur bullet positions into the wrong grid cells, which would be catastrophic for a controller that needs precise dodging.
The reconstruction image in this section shows a representative sample from this visual evaluation process. If the reconstructed grids fail to preserve the spatial arrangement of the original grids, the latent vector is not suitable for downstream NEAT training.
Why the CNN Input Was Necessary
The raw 2050-dimensional observation created a credit-assignment nightmare for NEAT. In a typical frame, over 95% of the danger grid cells are zero, meaning most genome weights receive no meaningful gradient signal. The CNN bottleneck compresses the most salient features into 66 dimensions (64 latent + 2 position), reducing the search space by 30x and eliminating the noise from empty grid cells.
However, this compression introduces a new limitation: the 64-dimensional latent vector is a flat representation with no preserved spatial structure. Although the encoder produces a 64x8x8 feature map internally, the final Linear layer mixes all spatial positions together before outputting the latent vector. The NEAT controller therefore receives 64 independent values that must be interpreted without explicit spatial layout, making it difficult to learn spatial relationships such as “bullets are approaching from the right side” directly from the latent representation.
The CNN-specific training UI visualizes the latent vector as an 8x8 heatmap for manual inspection, but this is purely a display convenience. The actual input to the NEAT genome is the flat 64-element vector plus two position values, totaling 66 inputs.
Training & Simulation Framework
The training pipeline orchestrates three main components:
- Game environment - TH06 running headless via the decompilation port, exposed over a Unix socket
- Observation encoder - converts game state into either raw 32×32 grids or CNN-latent vectors
- NEAT evolution loop - neuroevolutionary search over neural network controllers
The CnnNeatTrainer class manages the full training lifecycle:
- Initializes the NEAT configuration with proper input/output dimensions
- Launches the game instance and maintains a persistent process across genomes
- Runs evaluation episodes, collecting reward components and survival statistics
- Supports checkpoints and saving the best genome per generation
- Integrates the CNN autoencoder for latent observation encoding
The trainer factory methods that wire up the CNN encoder and UI:
101def cnn_encoder_factory(config):
102 if config.dataset_directory is not None:
103 config.dataset_directory = PROJECT_DIRECTORY / "dataset"
104 encoder = CnnObservationEncoder(config, WEIGHTS_PATH, use_cpu=force_cpu)
105 encoder_holder["encoder"] = encoder
106 return encoder
107
108
109def cnn_ui_factory(*args, **kwargs):
110 ui = CnnTrainingUi(*args, **kwargs)
111 encoder_holder["encoder"].set_ui(ui)
112 return ui
The observation encoder transforms raw game state into the appropriate input format. The CnnObservationEncoder from code/cnn_encoder.py performs this transformation:
39 @property
40 def dim(self) -> int:
41 return self.latent_dim + 2
42
43 def set_ui(self, ui: CnnTrainingUi) -> None:
44 self.ui = ui
45
46 def reset(self) -> None:
47 self.raw_encoder.reset()
48
49 def flush_dataset(self) -> None:
50 self.raw_encoder.flush_dataset()
51
52 def encode(self, state: StateMessage, previous_action: int = 0) -> list[float]:
53 raw_observation = self.raw_encoder.encode(state, previous_action)
54 raw_grids = np.asarray(raw_observation[:-2], dtype=np.float32).reshape(1, 2, 32, 32)
55 inputs = torch.from_numpy(np.clip(raw_grids, 0.0, 1.0)).to(self.device)
56 with torch.inference_mode():
57 latent = self.model.encoder(inputs)[0].cpu().tolist()
58 if self.ui is not None:
59 self.ui.set_raw_observation(raw_observation[:-2])
60 self.ui.set_cnn_latent(latent)
61 return latent + raw_observation[-2:]The CNN-specific UI panel renders the raw observation grids and the latent heatmap:
44 def draw(self, generation: int, best_genome=None, current_genome: int | None = None) -> None:
45 super().draw(generation + self.generation_offset, best_genome, current_genome)
46
47 def _observation_panel(self, rect: pygame.Rect) -> None:
48 pygame.draw.rect(self.screen, (24, 30, 45), rect)
49 self._text("CNN INPUT / LATENT", (rect.x + 10, rect.y + 10), (225, 232, 245), self.small_font)
50 if self.raw_observation is None:
51 self._text("waiting for observation", (rect.x + 10, rect.y + 34), (130, 145, 165), self.small_font)
52 return
53 size = min(280, (rect.width - 68) // 2)
54 gap = 32
55 start_x = rect.x + (rect.width - 2 * size - gap) // 2
56 for layer, (label, color) in enumerate((("danger path", (255, 75, 65)), ("powerups", (85, 170, 255)))):
57 x = start_x + layer * (size + gap)
58 self._text(label, (x, rect.y + 20), color, self.small_font)
59 self._grid(pygame.Rect(x, rect.y + 42, size, size), self.raw_observation[layer * 1024:(layer + 1) * 1024], color)
60 self._text(f"grid samples collected: {self.dataset_samples:,}", (start_x, rect.y + 330),
61 (180, 195, 220), self.small_font)
62 self._text("CNN latent (8x8)", (start_x, rect.y + 350), (170, 190, 255), self.small_font)
63 if self.cnn_latent is not None:
64 self._latent_grid(pygame.Rect(start_x + 140, rect.y + 342, 80, 80), self.cnn_latent)
Baseline Controllers
Before NEAT evolution, several baseline controllers were implemented to establish performance benchmarks. These are defined in code/controllers.py:
11class NoOpController:
12 """Controller that takes no action (all inputs 0)."""
13
14 def act(self, state: StateMessage) -> int:
15 return 0
16
17
18class FixedCenterController:
19 """
20 Controller that moves the player to the center of the screen.
21
22 Reads current player position and issues input to move towards center.
23 """
24
25 def __init__(self, center_x: float = 192.0, center_y: float = 224.0):
26 self.center_x = center_x
27 self.center_y = center_y
28
29 def act(self, state: StateMessage) -> int:
30 dx = self.center_x - state.player_x
31 dy = self.center_y - state.player_y
32
33 action = 0
34 if abs(dx) > 5:
35 action |= 0x02 if dx > 0 else 0x01 # right = 2, left = 1
36 if abs(dy) > 5:
37 action |= 0x08 if dy > 0 else 0x04 # down = 8, up = 4
38 return action
39
40
41class RandomController:
42 """Controller that selects a random action (for baseline)."""
43
44 def __init__(self, seed: int | None = None):
45 import random
46
47 self.random = random.Random(seed)
48
49 def act(self, state: StateMessage) -> int:
50 return self.random.randint(0, 255)
These controllers served as negative controls and helped diagnose whether the NEAT-evolved genomes were actually learning meaningful behavior.
Reward Function Design & Iterative Tuning
A key aspect of this project was the iterative refinement of the reward function across multiple development phases. The reward components in th06_env.py show clear phase-based evolution, reflecting an ongoing process of fine-tuning based on empirical results. The RewardComponents class tracks each term separately:
1class RewardComponents:
2 """Tracks individual reward components for transparent logging."""
3
4 __slots__ = ("survival", "score_delta", "graze_delta", "powerup", "focus_danger",
5 "bomb_misuse", "edge_hugging", "death_penalty", "edge_termination", "total")
The final configuration after three phases of tuning:
1# Survival reward (always constant)
2SURVIVAL_REWARD: float = 0.01
3
4# Score-based rewards
5SCORE_REWARD_DIVISOR: float = 1000000.0
6GRAZE_REWARD: float = 100.0
7POWERUP_REWARD: float = 350.0
8FOCUS_DANGER_REWARD: float = 5.0
9FOCUS_DANGER_RADIUS_MULTIPLIER: float = 8.0
10
11# Penalty terms (showing phase evolution)
12BOMB_MISUSE_PENALTY: float = 350.0 # Phase 2: 150.0 # Phase 1: 50.0
13BOMB_DANGER_BULLET_THRESHOLD: int = 3
14
15EDGE_HUGGING_PENALTY: float = 100.0 # Phase 3: 10.0 # Phase 2: 5.0 # Phase 1: 2.0
16EDGE_HUGGING_MARGIN: float = 48.0 # Phase 2: 24.0
17EDGE_HUGGING_THRESHOLD: int = 6000 # Phase 2: 6000000000
18EDGE_HUGGING_TERMINATION_PENALTY: float = 5000.0
19
20DEATH_PENALTY: float = 25000.0
21MAX_FRAMES_PER_EPISODE: int = 12000 # Phase 2: 6000
Phase 1 (Initial Experiments)
Extremely lenient penalties: Bomb misuse at -50, edge hugging at -2/frame. Very short episodes at 6,000 frames maximum. The goal was simply to establish the basic framework and verify NEAT could learn anything at all. At this stage, the genomes were dying almost immediately from bomb spam or walking into corners, but the signal was too weak to discourage these behaviors.
Phase 2 (Stabilization)
Moderate penalties: Bomb misuse increased to -150, edge hugging to -5/frame. Episode length doubled to 12,000 frames maximum. Edge hugging margin reduced from effectively infinite to 48 pixels (roughly 1/8th of the playfield width). The goal was to discourage obvious maladaptive behaviors while still rewarding exploration. This helped, but genomes still tended to bomb too early or hug edges when under pressure.
Phase 3 (Fine-tuning)
Strong penalties: Bomb misuse at -350, edge hugging at -100/frame. Continued long episodes with refined edge detection. The goal was to create a clear survival signal where even small mistakes are heavily penalized. The edge hugging threshold of 6000 frames means if the player stays within 48 pixels of any screen edge for that many consecutive frames (about 100 seconds at 60 FPS), the episode terminates with a -5000 penalty.
The bomb misuse logic checks if the player presses bomb when fewer than 3 bullets are within the focus danger radius (player hitbox 1.25 × 8.0 = 10 pixels). This prevents wasting bombs on sparse patterns while still allowing them for dense waves.
The focus danger reward gives +5 per frame when holding focus (shift) and at least one bullet is within the danger radius. This encourages the AI to use focused movement for precision dodging rather than just bombing.
The death penalty at -25,000 is deliberately massive compared to any positive reward. A single death wipes out 250,000 frames of survival reward, or roughly 70 powerup collections. This forces the NEAT population to prioritize survival above all else.
This evolution reflects the fundamental challenge in sparse-reward environments like Touhou 6: the agent receives meaningful feedback only for rare events (collecting powerups: +350, grazing bullets: +100) or catastrophic events (death: -25,000). Most frames provide only the minimal survival reward (+0.01), making credit assignment difficult.
The NEAT training configuration for the CNN-preprocessed variant reflects these insights, using a population size of 200 to maintain diversity while searching for genomes that can navigate this sparse reward landscape.
Results
Despite experimentation with both full-grid and CNN-preprocessed NEAT, the AI was unable to play Touhou 6 above random performance levels. The final generation (generation 49) of the CNN-preprocessed run produced the following metrics:
| Metric | Value |
|---|---|
| Best fitness | -16,142 |
| Average fitness | -171,165 |
| Best score | 1,256,350 |
| Average score | 214,733 |
| Best survival frames | ~2,599 |
| Best powerups collected | 23 |
| Best grazes | 6 |
| Episodes completed | 0 |
The best genome survived approximately 2,599 frames (25.99 survival reward / 0.01 per frame), collected 23 powerups (8,050 reward / 350 each), and grazed 6 bullets (600 reward / 100 each). However, it still died, incurring the full -25,000 death penalty.
Across all 150 evaluated genomes, the average reward components show the core problems:
| Reward Component | Average Value |
|---|---|
| Survival | +20.98 |
| Graze | -3.33 |
| Powerups | +7,170 |
| Focus danger | +134 |
| Bomb misuse | -71,888 |
| Edge hugging | -81,600 |
| Death penalty | -25,000 |
The bomb misuse and edge hugging penalties dominated the average reward, showing that the evolved genomes were repeatedly bombing too early and hiding near screen edges. Every genome died, which is why the best fitness remained negative despite the positive rewards from powerups, graze, and focus.
The CNN preprocessing did improve consistency compared to the full-grid variant, but the reward signal proved too sparse for effective credit assignment within the limited training budget (one weekend, 50 generations, population 200).
Baseline controllers performed as expected:
- NoOpController: Survival ~200 frames, score ~0
- FixedCenterController: Survival ~300 frames, modest graze rewards
- RandomController: Survival ~150 frames, erratic behavior
The full training logs and per-generation statistics are available in the neat_output/ directory of the repository.
Reproducing the Project
If you want to run this project yourself or build on it, here are the practical setup details from the repository:
Game Data (Required)
The decompilation project requires assets from a legally owned copy of Touhou 6: Embodiment of Scarlet Devil. Copy these into the repository root:
- BGM directory:
bgm/containing.wavor.midfiles - Game data files (all required):
紅魔郷CM.DAT紅魔郷ED.DAT紅魔郷IN.DAT紅魔郷MD.DAT紅魔郷ST.DAT紅魔郷TL.DAT
- Config file (optional):
東方紅魔郷.cfg - Font: A compatible Japanese font at
msgothic.ttc(MS Gothic or Kochi Gothic)
The project cannot run without these files - the C++ binary loads them at startup.
C++ Build Process
The game engine is built from the modified decompilation source using Premake5 and Make:
1# Install system dependencies (Debian/Ubuntu)
2sudo apt install build-essential libsdl2-dev libsdl2-image-dev libsdl2-ttf-dev libasound2-dev
3
4# Generate build files and compile
5premake5 gmake && cd build && make -j16 && cd ..
The th06 binary will appear in the repository root. This binary includes the AI input socket, headless mode, and frameskip acceleration needed for training.
Python Dependencies
1python/.venv/bin/python -m pip install -r python/requirements.txt
Key dependencies: NEAT-Python, PyTorch, pygame, numpy.
Training Pipeline
The full pipeline has four stages:
-
Collect dataset - Run full-grid NEAT with dataset saving to generate
dataset/chunk_*.npzfiles:1python/.venv/bin/python launch_instances.py --neat --frameskip 10000 --no-brain --quiet -
Train CNN autoencoder - Compress the 32×32 grids to 64-dim latent vectors:
1cd python/cnn 2../.venv/bin/python trainer.pyWeights saved to
python/cnn/weights/grid_autoencoder.pt. -
Inspect reconstructions - Verify the autoencoder preserves spatial details:
1cd python/cnn 2../.venv/bin/python view_autoencoder.pyKeyboard:
Right/Spacenext,Leftprevious,Rrandom,Etest empty sample,Qquit. -
Train CNN-NEAT - Headless training with live gnuplot reward plots:
1cd python/cnn-train 2../.venv/bin/python launch_instances.py --neat --frameskip 10000 --quiet --no-dataset-save --cpu --no-uiIn a separate terminal:
1gnuplot -persist plot.gp
For a full-grid NEAT baseline (without CNN), omit the --cpu flag and run from the main python directory.
Why the Project Failed
The project failed for two interconnected reasons:
-
Excessive corner-hugging behavior: Despite the edge hugging penalty, the AI learned that staying in corners minimized immediate danger from bullets. The edge detection system penalized proximity to edges but did not distinguish between active dodging (moving along edges to avoid patterns) and passive hiding (remaining stationary in corners). Over long episodes, the small survival reward (+0.01 per frame) accumulated to outweigh the periodic edge hugging penalties, especially when the AI could survive tens of thousands of frames in a corner before penalties became significant.
-
Insufficient spatial detail in CNN grids: The 32×32 input grids lacked the resolution needed for precise bullet dodging. Each grid cell represented approximately 12×12 pixels of the actual game screen. This coarse granularity made it impossible for the CNN to learn fine-grained patterns of bullet trajectories. The autoencoder’s latent space, while compressing the most active features, could not recover the spatial precision lost in the initial downsampling to 32×32. Consequently, the NEAT controller received observation vectors that lacked the detail necessary to distinguish between safe and unsafe micro-movements within a grid cell.
These two factors combined to create a local optimum where corner hugging with occasional focusing emerged as the highest-reward strategy, but the AI was inadequate for actual Touhou 6 gameplay which requires precise movement through complex bullet patterns.
Next Steps
The attempt, while unsuccessful in achieving playable AI, revealed several insights:
- CNN preprocessing aids learning - reducing 2050 inputs to 66 latent + position values made training more stable and faster
- Reward sparsity is the primary bottleneck - the game provides meaningful rewards only for graze (100), powerup collection (350), and death events; intermediate gameplay yields minimal signal
- Population size matters - 200 genomes provided diversity but insufficient for the credit-assignment problem within limited training time
- Full-grid NEAT struggles more - the raw 2050-dimensional observation creates a credit-assignment nightmare; the CNN bottleneck actually helps
- Iterative reward shaping is essential - the phased evolution of penalty terms (particularly for bomb misuse and edge hugging) was critical for making any progress at all
- Spatial resolution is critical for bullet dodging - the 32×32 grid representation loses too much precision for pattern recognition
- Latent space interpretability matters - the 64-dimensional latent vector, while computationally efficient, obscures the spatial relationships NEAT needs to evolve effective dodging strategies
Future directions include:
- Curriculum learning - start with simpler configurations (fewer bullets, reduced screen coverage) and gradually increase complexity
- Shaped reward functions - intermediate rewards for near-misses, focused dodging, or pattern recognition
- Hybrid approaches - combine NEAT with policy gradient methods for continuous control aspects
- Deep Q-Learning (DQN) - replace neuroevolution with value-based deep RL, which is better suited for fixed discrete action spaces and provides stronger credit assignment through temporal difference learning
- Longer training horizons - multiple weeks of training with larger populations and checkpoint resumption
- Enhanced observation encoding - higher resolution grids or feature pyramids to preserve spatial detail
- Temporal context - frame stacking or recurrent connections to capture motion patterns and velocity information
The project remains a valuable exploration of the intersection between classical neuroevolution and modern deep learning preprocessing for classic game AI.
