pylos (opens in a new tab)
Play Pylos against a transformer trained by self-play reinforcement learning.
How it works
Each player has 15 balls, and together they build a pyramid of 30. On your turn, place a ball from your reserve in an empty hollow or on a square of four balls. You may instead raise one of your balls onto a square, if nothing rests on it, which saves a ball. Complete a square of four balls of your colour and you take back one or two of your balls that support nothing. Whoever places the top ball wins, and a player whose reserve is empty at the end of their turn loses.
A transformer of 0.4 million parameters reads the 30 cells and both reserves. It gives a probability to every legal move and estimates its chance of winning. It learned only by playing against itself: 139,000 games in 20 minutes on a $350 GPU. Before each move, a short search (Gumbel AlphaZero) improved on its own move probabilities, and the model learned to predict the improved moves and the result of each game.
Here it looks at 64 positions before each move. Against an alpha-beta search three turns deep, it won 383 of 400 games. The rules, the model and its search are one Rust program, used for training and compiled to WebAssembly for this page.