A Simple Transformer Solves Conway's Game of Life
2025-05-17
Researchers have shown that a highly simplified transformer neural network can perfectly compute Conway's Game of Life solely by training on examples of the game. The model uses its attention mechanism to effectively compute 3x3 convolutions, mirroring the neighbor-counting crucial to the Game of Life's rules. Called SingleAttentionNet, its simple structure allows for observation of its internal computations, demonstrating it's not a simple statistical predictor. The study reveals the model can perfectly run 100 games for 100 steps, even when trained only on the first and second iterations of random Game of Life instances.
Read more