“Reflexion: Language Agents with Verbal Reinforcement Learning” was posted to arXiv on March 20, 2023 by Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. It proposed a way for a language agent to improve across attempts without changing any of its weights, using written self-criticism in place of gradient-based reinforcement learning.
The mechanism is a loop. The agent attempts a task, receives feedback - which can be a numeric score, an error message, or open-ended language, from an external source or generated by the model itself - and then writes a verbal reflection on what went wrong. That reflection is stored in an episodic memory buffer and fed back in on the next attempt, so the agent reasons about its past failures before trying again. Because the “learning” lives in text rather than in updated parameters, it is cheap and immediate. The headline result was 91 percent pass@1 on the HumanEval coding benchmark, above the 80 percent the authors cited for GPT-4 at the time.
Reflexion, from some of the same authors as ReAct, became the canonical reference for the reflection pattern that Andrew Ng later named as one of the four agentic design patterns. It is a clean demonstration that giving an agent a way to remember and reason about its own mistakes can lift performance without any retraining.