On February 16, 2022, DeepMind announced that, working with the Swiss Plasma Center at EPFL in Lausanne, it had used deep reinforcement learning to control the plasma inside a real nuclear-fusion device. The work was published in Nature as “Magnetic control of tokamak plasmas through deep reinforcement learning.”
Tokamaks confine plasma at extreme temperatures using magnetic fields, and holding that plasma in a stable shape normally requires separately tuned controllers for each magnetic coil. DeepMind instead trained a single neural network in simulation to drive all of the Variable Configuration Tokamak’s 19 control coils at once, then deployed it on the physical machine.
The learned controller produced and held a range of plasma configurations on the TCV, including conventional elongated shapes, advanced shapes such as negative-triangularity and “snowflake” configurations meant to spread exhaust heat, and a shape matching proposals for the much larger ITER reactor. It even sustained “droplets,” two separate plasmas held in the vessel at once, which DeepMind said had never been done before on that machine.
This was one of the hardest real-world control problems reinforcement learning had been applied to, and an early sign that learned controllers could accelerate experimental physics rather than only winning games or generating text.