order flow

Reinforcement learning in a market simulated by a transformer trained on Nasdaq order flow.

Loading the models (3 MB)…

How it works

A transformer of 3.4 million parameters reads the orders sent to a stock exchange the way a language model reads text, and predicts the next one: a buy, a sell or a cancellation, its price, its size and how soon it comes. It was trained on 13 days of Nasdaq orders.

Here it plays 24 traders in a simulated exchange that starts from a real order book. Some follow the model as it is. Others tilt it towards their own estimate of the value, towards recent moves, or towards the holdings they started with. News moves those estimates, and uncertainty makes traders cancel more. The price comes only from the orders the exchange matches.

The trader drawn in colour decides every 20 orders whether to wait, cancel, buy or sell. Its choices start from the model's probabilities. Reinforcement learning trained a small layer on top of the model to shift them, so that its account grows while it holds few shares. On days kept out of training, it ended $13 to $18 ahead of keeping its shares.