“Magnetic control of tokamak plasmas through deep reinforcement learning,” by Jonas Degrave, Federico Felici and colleagues at DeepMind and the Swiss Plasma Center at EPFL, was published in Nature in February 2022. It reported the first use of reinforcement learning to directly control the plasma inside a working tokamak, a doughnut-shaped device that confines superheated plasma with magnetic fields in pursuit of nuclear fusion energy.
In a tokamak, dozens of magnetic coils must be adjusted thousands of times a second to hold the plasma in shape and keep it from touching the vessel walls. Traditionally each coil is governed by separately designed controllers. The DeepMind and EPFL team instead trained a single neural network in simulation to command all of the control coils of the Variable Configuration Tokamak (TCV) in Lausanne, then deployed it on the real machine. The learned controller, taking only magnetic measurements as input, autonomously produced and held a range of plasma configurations, including elongated shapes, a “snowflake” configuration, and a droplet with two separate plasmas at once.
Fusion offers the promise of abundant clean energy, and plasma control is one of its central engineering challenges. By showing that a learned controller could match goals just by being told the target shape, the work pointed to a faster way to design and test the control systems that future reactors such as ITER will need.