Why Everyone is Rushing to Build Reinforcement Learning Environments

Building reinforcement learning (RL) environments is quickly emerging as the next big thing in AI. OpenAI co-founder Andrej Karpathy recently noted in his post on X that the evolution of AI training can be broken down into three distinct eras—pretraining, supervised finetuning and, now, reinforcement learning environments. “In the era of pretraining, what mattered was internet text,” Karpathy explained. The priority then was to gather a large, diverse and high-quality collection of online documents to train models. With supervised finetuning, the focus shifted to conversations. “Contract workers were hired to create answers for questions, a bit like what you’d see on Stack Overflow or Quora, but geared towards LLM use cases,” he said. According to Karpathy, neither of